-- Logs begin at Thu 2022-09-15 12:17:11 UTC, end at Thu 2022-09-15 13:24:36 UTC. -- Sep 15 12:44:02.716159 dv-cinder-836083-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 15 12:44:02.724025 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 15 12:44:02 2022] *** Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: nodename: dv-cinder-836083-13-iscsi Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: machine: x86_64 Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: clock source: unix Sep 15 12:44:02.726050 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: pcre jit disabled Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: detected number of CPU cores: 4 Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: current working directory: / Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: detected binary path: /usr/bin/uwsgi-core Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: your processes number limit is 47680 Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: your memory page size is 4096 bytes Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: detected max file descriptor number: 2048 Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: lock engine: pthread robust mutexes Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: thunder lock: enabled Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 15 12:44:02.726492 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 15 12:44:02.767001 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: Python main interpreter initialized at 0x558c673c29c0 Sep 15 12:44:02.767001 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: python threads support enabled Sep 15 12:44:02.767001 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: your server socket listen backlog is limited to 100 connections Sep 15 12:44:02.767001 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: your mercy for graceful operations on workers is 90 seconds Sep 15 12:44:02.767401 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: mapped 403077 bytes (393 KB) for 2 cores Sep 15 12:44:02.767485 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: *** Operational MODE: preforking *** Sep 15 12:44:02.767727 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: *** uWSGI is running in multiple interpreter mode *** Sep 15 12:44:02.767727 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: spawned uWSGI master process (pid: 81392) Sep 15 12:44:02.767930 dv-cinder-836083-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 15 12:44:02.769044 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: spawned uWSGI worker 1 (pid: 81406, cores: 1) Sep 15 12:44:02.769044 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: spawned uWSGI worker 2 (pid: 81407, cores: 1) Sep 15 12:44:02.769044 dv-cinder-836083-13-iscsi devstack@keystone.service[81392]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 15 12:44:07.065280 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:44:07.065280 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Sep 15 12:44:07.066087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:44:07.066087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Sep 15 12:44:07.070374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:44:07.070374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: from cryptography.utils import int_from_bytes Sep 15 12:44:07.070838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:44:07.070838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: from cryptography.utils import int_from_bytes Sep 15 12:44:07.116441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 12:44:07.116632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 12:44:07.116823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] command line args: [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 12:44:07.117088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 12:44:07.117262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ================================================================================ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 12:44:07.117514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.117758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_dir = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.117835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 12:44:07.117942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.118021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 12:44:07.118119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_source = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.118208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] command line args: [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 12:44:07.118280 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] debug = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.118458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 12:44:07.118551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.118717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ================================================================================ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 12:44:07.118890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.118980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] admin_token = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_dir = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_source = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] debug = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.119938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_config_append = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_dir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.120999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_config_append = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_dir = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.121982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122277 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.122996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.123115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.123195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.123306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.123411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.123536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.124292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.124430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.124613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.124870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] publish_errors = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.125884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] publish_errors = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] standard_threads = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.126901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_journal = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_json = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] standard_threads = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.127963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_stderr = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_syslog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.128952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_journal = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.128952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_json = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.129172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.129285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_stderr = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.129396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.129508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_syslog = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.129628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.129742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:44:07.129855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.129967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.130878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.131917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.132993 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.133906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.134938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.135751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136242 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.136988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.137895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.138925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.139039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.139039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.139250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.139984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.140880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.141936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.142914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.143962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144479 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.144962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.145903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.146975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.147913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.148978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.149902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.150943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.151940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.152888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.153967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.154935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.external = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.155917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.external = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.password = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.token = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.156939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.password = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.token = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.157890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.158943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.159926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.160911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.161954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.162878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.163928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.164995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.165889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.166818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.167965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.168911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.169899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.170975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.171907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.172940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.173889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.174946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.175988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.176946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.177995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.178921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.179972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.180923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.181969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.182911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.183973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.184965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.185893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.186008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.186655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.186785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.186902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.187930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.188883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.189990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.190918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.191969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.192958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.193899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.194910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.195970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.196924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.197970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.198907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.199834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.200806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.201037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.201037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.201261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.201385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.201502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.201625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.202343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.202501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.202674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.202796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.202945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.203919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.204878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.205936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.206973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.207097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.207214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:44:07.207335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 12:44:07.207451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 12:44:07.234398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.234623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.234848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.238609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.238843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.239064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.243180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.247414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.272840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.273063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.274957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.275192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.328391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.329592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.382734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.382965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.383798 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.384036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.539640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.539873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.540929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.541140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.564650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.564883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.564969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.565965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.566043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.566171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.577133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.577521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.589958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.590997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.591077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.591207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.591287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.591413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:44:07.607837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.608602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:44:07.608948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.609146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:44:07.609231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.609415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:44:07.609498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.609671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.609754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.609926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.610019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:44:07.610182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.610265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.610445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.610701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.610955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.619635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.619915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.620188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.620443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.620527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.620722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.620992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.621275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.627956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.629065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.630844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.631130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.631416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.631701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.631907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.631991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.632192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.632305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.632497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.632579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.632772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.632867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.633835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.634103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.634371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.634634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.656427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.657834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.657923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.658150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.659269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.659553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.662749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.663037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.663319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.664132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.664439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.664718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.669591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.669876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.671680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.671970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.676294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.676741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.677172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.677325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.677684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.677955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.678046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.678255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.678382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.678526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.678675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.678863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.679198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.680034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.680034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.680034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.694710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.694988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.696297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.696576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.698857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.700376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.702075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.702368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.703978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.704282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.706517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.708443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.709733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.711647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.712935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.713235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.713523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.713807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.714081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.714373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.714823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.715120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.715405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.715694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.715982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.716291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.729933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.730219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.730489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.730755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.733952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.734246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.734519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.734786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.738246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.739781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.741319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.741598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.741682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.743124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.744338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.744627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.744719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.744917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.745000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.745202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.745599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.747604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.747893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.748178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.748491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.749671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.757340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.757630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.757912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.758195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.758575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.758868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.759149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.760708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.761250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.761532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.761817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.762201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.762492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.762772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.774616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.777517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.777997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.780896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.781390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.782319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.783884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.784168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.784252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.784588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.785718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.787183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.787469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.787866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.790840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.792294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.794178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.795146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.796074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.796339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.796428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.796666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.799157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.799559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.799852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.800136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.804343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.804637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.804919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.807853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.808152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.808466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.811715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.812002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.812302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.812582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.812854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.813126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.813400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.813700 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.813972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.814250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.814528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.814796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.815062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.815332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.815422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:44:07.815683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.815969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.816251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.816568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.816841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.817116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.817406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.817680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.817960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.818310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.818580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.818849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.819132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:44:07.849499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558c673c29c0 pid: 81407 (default app) Sep 15 12:44:07.852734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef17d2-ef14-42b4-91dc-91d7879f14f3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:07.853027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef17d2-ef14-42b4-91dc-91d7879f14f3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:07.853305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef17d2-ef14-42b4-91dc-91d7879f14f3 None None] PATH_INFO: `/v3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:07.853589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558c673c29c0 pid: 81406 (default app) Sep 15 12:44:07.857459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu Sep 15 12:44:07 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:14.614180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:14.614702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:14.614702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:14.618927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:14.624942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:14.625385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:14.625527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:14.696367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: INFO dbcounter [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] Registered counter for database keystone Sep 15 12:44:14.715897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG oslo_db.sqlalchemy.engines [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81407) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 12:44:14.722344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writer thread running {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 12:44:14.901645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG passlib.handlers.bcrypt [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81407) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 12:44:14.901955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG passlib.handlers.bcrypt [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81407) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 12:44:14.960083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:14.961721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:15.095624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 15 12:44:15.565270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:15.636824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: INFO dbcounter [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] Registered counter for database keystone Sep 15 12:44:15.658764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG oslo_db.sqlalchemy.engines [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 12:44:15.664918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writer thread running {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 12:44:15.783746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494c685-22ad-477b-99bf-4f0a3c8ecc01 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:15.788012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:15.788663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:15.788946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:15.789218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:15.798462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:15.850589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 12:44:15.850883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565a628-8029-451b-9855-48918df0a36e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:15.861158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu Sep 15 12:44:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:44:15.867067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:15.884585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5840fcc-8ab5-4cd9-b859-cb1389595a56 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:15.889869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:15.891003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:15.891284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:15.891553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] PATH_INFO: `/v3/projects/53ac1f185c9948448574c644790cea74` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:15.893148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] RBAC: Authorizing `identity:get_project(project_id=53ac1f185c9948448574c644790cea74)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:15.948410 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 12:44:15.948752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effc161-cc9c-4451-be9f-776badd4987b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:15.953637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu Sep 15 12:44:15 2022] GET /identity/v3/projects/53ac1f185c9948448574c644790cea74 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:18.078789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035efac-99e2-46a9-a7b8-f8ca38bb2b61 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:18.079408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035efac-99e2-46a9-a7b8-f8ca38bb2b61 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:18.079408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035efac-99e2-46a9-a7b8-f8ca38bb2b61 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:18.083427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:18.088887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:18.089330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:18.089447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:18.109928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:18.111438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:18.119347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 15 12:44:18.394030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035efac-99e2-46a9-a7b8-f8ca38bb2b61 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:18.406835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035efac-99e2-46a9-a7b8-f8ca38bb2b61 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:18.411076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec6631-8003-4475-8fcb-ed75125bd029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:18.411676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec6631-8003-4475-8fcb-ed75125bd029 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:18.411960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec6631-8003-4475-8fcb-ed75125bd029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:18.412238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec6631-8003-4475-8fcb-ed75125bd029 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:18.419987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec6631-8003-4475-8fcb-ed75125bd029 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:18.421795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec6631-8003-4475-8fcb-ed75125bd029 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:18.431368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu Sep 15 12:44:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:44:18.436607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:18.445583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb720f93-6277-45c0-9aee-1e2bbce3e37a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:18.449883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5c974-3f6f-42d2-912e-3403df53307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:18.450508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5c974-3f6f-42d2-912e-3403df53307e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:18.450790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5c974-3f6f-42d2-912e-3403df53307e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:18.451064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5c974-3f6f-42d2-912e-3403df53307e admin admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:18.452487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5c974-3f6f-42d2-912e-3403df53307e admin admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:18.454308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5c974-3f6f-42d2-912e-3403df53307e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:18.455813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu Sep 15 12:44:18 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:20.723106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e7021-c26c-4b0b-9864-de0802eebcc7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:20.723646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e7021-c26c-4b0b-9864-de0802eebcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:20.723646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e7021-c26c-4b0b-9864-de0802eebcc7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:20.725010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:20.730328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:20.730734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:20.731004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:20.749599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:20.750809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:20.751638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1a43-e5a8-446c-b093-1c4f2c82e35c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:20.751925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1a43-e5a8-446c-b093-1c4f2c82e35c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:20.752199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a1a43-e5a8-446c-b093-1c4f2c82e35c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:20.753542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:20.758654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:20.758941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:20.759219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:20.801654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81406) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 12:44:20.802040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81406) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 12:44:20.836448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:20.851394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:20.853017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:20.954909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:21.111504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.137142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.219030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d3e5d-ed11-41e2-8cf0-d799d3e572c1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.222918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.252928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.253565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.253844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.254121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.265603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:21.274392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:21.275469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:21.276483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:21.283670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.325000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312408a-74f5-4b7f-9310-5f52718875a6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.327297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.363615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.364437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.364725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.364957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.365051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.382720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2d92-6e7b-4d6c-b155-eec183832912 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.416174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.416808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.417090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.417371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.418040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:21.418345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.419738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.432104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:21.433564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.435077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.437929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.533698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398394c-e615-412c-9bf3-7fb48b232ab8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.534578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:44:21.540120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.542816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.567531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.568175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.568494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.568774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.569921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.571385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.572995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:21.578437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.592087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.639044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5c769-4093-4a54-a636-6ac8ac35ae7d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.673197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.674074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.674356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.674629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.678262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76eab-0eb9-47fc-a1d7-1aaa2bda7ff6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.693265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.694738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.712746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.713375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.713651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:44:21.713746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:21.714677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:44:21.715451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:44:21.715451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:44:21.715451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:21.715602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.715670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 9/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:44:21.715731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.716478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.718577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 10/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:21.722294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.724210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:21.778843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.804919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:21.823654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7461925-388e-466f-8222-b924be1608b8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.826092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5eb4d-8e5b-4790-b606-001e8200849d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:21.858756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.859350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.859628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.859898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.860668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.domain.id=default, user.id=07cb4dfbc4ad477d912360d9e55fab21) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:21.860970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.861440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:21.862055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:21.862336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:21.862415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72aa6b1-d000-46e0-a02e-5f0f14419c62 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.862604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:21.863238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:21.863542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:21.864943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac2571-3f70-450d-9cd9-d5010fb26851 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:21.869543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/role_assignments?user.id=07cb4dfbc4ad477d912360d9e55fab21&scope.domain.id=default&role.id=13d4313b2fe74b3688999f3077afce5d => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:21.883315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:24.279064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8846f-202d-4c37-b15a-55052756cba3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:24.280193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8846f-202d-4c37-b15a-55052756cba3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:24.280719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8846f-202d-4c37-b15a-55052756cba3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:24.282574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:24.288655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be665b-8d5b-4b39-af5d-65f9e45a12dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:24.289647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be665b-8d5b-4b39-af5d-65f9e45a12dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:24.290175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be665b-8d5b-4b39-af5d-65f9e45a12dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:24.317296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-29be665b-8d5b-4b39-af5d-65f9e45a12dc None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:24.318853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-29be665b-8d5b-4b39-af5d-65f9e45a12dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:24.396655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c953a-9eae-4845-aebb-2e6a82fa9fd7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:24.397344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c953a-9eae-4845-aebb-2e6a82fa9fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:24.397829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c953a-9eae-4845-aebb-2e6a82fa9fd7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:24.399347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 13/23] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:24.404934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c02e96-a895-4c8f-9295-4fdb9feb2578 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:24.405545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c02e96-a895-4c8f-9295-4fdb9feb2578 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:24.406046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c02e96-a895-4c8f-9295-4fdb9feb2578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:24.413124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 11/24] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:24.425681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-40c02e96-a895-4c8f-9295-4fdb9feb2578 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:24.427151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-40c02e96-a895-4c8f-9295-4fdb9feb2578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:24.515542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:24.760245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2073853-a7eb-4949-8c70-3e7f2bc585eb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:24.761226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2073853-a7eb-4949-8c70-3e7f2bc585eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:24.761707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2073853-a7eb-4949-8c70-3e7f2bc585eb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:24.763272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 12/26] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:24.769241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:24.769789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:24.770270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:24.793207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:24.794732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:24.868425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2073853-a7eb-4949-8c70-3e7f2bc585eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:24.898918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 15/27] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:24.944791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.009041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2073853-a7eb-4949-8c70-3e7f2bc585eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.052539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.053104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.053383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.053655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.070128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:25.076734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:25.077619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:25.078361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:25.082257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 13/28] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:25.085360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.088000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.100460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.294164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74f28e-09e1-4bfa-80c3-422a82bca9dd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.306984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.318435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1df50-8dc8-44fc-ab3d-d7bbcb2fb595 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.346432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.347076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.347360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.347709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.359647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.361622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:44:25.378470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:25.379348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:44:25.380161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:44:25.380161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:44:25.380161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:25.381948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 14/29] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:25.391717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.404773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.405446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.405734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.406013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.406703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:25.407011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.416311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.418849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 16/30] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:25.419541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.535902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305f651-0be0-4a71-a94e-31b8e714475f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.561853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.584952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.586019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.586907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.587357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.588485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:25.589159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.591343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.599571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02807f9a-097f-41f7-9c8e-733334f50adf None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.607568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 15/31] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:25.609379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.635294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.637745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.638932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.639344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.639697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.640480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.642662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.731431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5ff37-0c36-4b5b-85ed-228945c4b193 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.767110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd00e4e-d999-4c68-8adb-6426be067d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.768040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd00e4e-d999-4c68-8adb-6426be067d4b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.768545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd00e4e-d999-4c68-8adb-6426be067d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.769015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd00e4e-d999-4c68-8adb-6426be067d4b None admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.770652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd00e4e-d999-4c68-8adb-6426be067d4b None admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.772320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd00e4e-d999-4c68-8adb-6426be067d4b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.774091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 16/32] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:25 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:25.775631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbd049-fb98-4fa8-ba21-98434ddce5b6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.776136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbd049-fb98-4fa8-ba21-98434ddce5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.776753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbd049-fb98-4fa8-ba21-98434ddce5b6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.778822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 17/33] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:25.779771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbd049-fb98-4fa8-ba21-98434ddce5b6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.789409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 17/34] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Sep 15 12:44:25 2022] POST /identity/v3/roles => generated 219 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:25.793679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:25.882313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.904744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbd049-fb98-4fa8-ba21-98434ddce5b6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.926522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e127db-59ec-4acb-a1c3-091fc357a3d4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:25.949639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44c0a9-c38a-41c3-a6f6-d7eadef9facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.950656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44c0a9-c38a-41c3-a6f6-d7eadef9facd None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.951682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44c0a9-c38a-41c3-a6f6-d7eadef9facd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.952800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44c0a9-c38a-41c3-a6f6-d7eadef9facd None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.954131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44c0a9-c38a-41c3-a6f6-d7eadef9facd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.955851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44c0a9-c38a-41c3-a6f6-d7eadef9facd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.958519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:25.971863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6bdd-a9a9-49b0-a0d0-cc4de53abe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:25.972820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6bdd-a9a9-49b0-a0d0-cc4de53abe1b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.973312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6bdd-a9a9-49b0-a0d0-cc4de53abe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.973821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6bdd-a9a9-49b0-a0d0-cc4de53abe1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:25.976079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6bdd-a9a9-49b0-a0d0-cc4de53abe1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:25.978197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6bdd-a9a9-49b0-a0d0-cc4de53abe1b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:25.984304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 18/35] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:25.984304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:25.984304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:25.984304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.010655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:26.013146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:26.102088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 19/36] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:26.102949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:26.188789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 18/37] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:44:25 2022] POST /identity/v3/projects => generated 301 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:26.190202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b057b9-5409-49e1-aba2-a893438fa701 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.190495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b057b9-5409-49e1-aba2-a893438fa701 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.190785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b057b9-5409-49e1-aba2-a893438fa701 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.192204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 19/38] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:26.198514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.198825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.199110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.222981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:26.224617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:26.233320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd17600-3064-44f8-8129-8a120ce5c32c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:26.274147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:26.291465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91f27b-8d6a-4b25-a4d1-6f7e8873fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:26.292156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91f27b-8d6a-4b25-a4d1-6f7e8873fab7 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.292471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91f27b-8d6a-4b25-a4d1-6f7e8873fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.292753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91f27b-8d6a-4b25-a4d1-6f7e8873fab7 None admin] PATH_INFO: `/v3/domains/default/users/07cb4dfbc4ad477d912360d9e55fab21/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.294819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91f27b-8d6a-4b25-a4d1-6f7e8873fab7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=07cb4dfbc4ad477d912360d9e55fab21, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:26.296758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91f27b-8d6a-4b25-a4d1-6f7e8873fab7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:26.318249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 20/39] 10.222.0.57 () {58 vars in 1361 bytes} [Thu Sep 15 12:44:26 2022] PUT /identity/v3/domains/default/users/07cb4dfbc4ad477d912360d9e55fab21/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:26.319894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db181cd-79a6-4c47-ab80-4b88749b3c60 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.320476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db181cd-79a6-4c47-ab80-4b88749b3c60 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.320563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db181cd-79a6-4c47-ab80-4b88749b3c60 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.322317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 21/40] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:26.329554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.330178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.330563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.335666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 20/41] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:26.354360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:26.355794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:26.449865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 22/42] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:26.527575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:26.605537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:26.641738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395efd3-aec6-4700-b663-92f20d2c7fa1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:26.680389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:26.681013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.681296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.681571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.682240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:26.683848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:26.688957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:26.707808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 21/43] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Sep 15 12:44:26 2022] POST /identity/v3/roles => generated 225 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:26.745688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:26.829983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ea5fd-99bf-4476-8092-45339f290ba8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:26.848057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:26.877149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:26.877789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.878183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.878464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.879156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:26.882455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:26.882620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08ad0d-13b9-4536-8baf-6351b8e76297 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:26.898307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 23/44] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Sep 15 12:44:26 2022] POST /identity/v3/roles => generated 223 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:44:26.920426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:26.929552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d75bb-790d-4d51-b1e5-b50ff770f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:26.930508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d75bb-790d-4d51-b1e5-b50ff770f836 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:26.930945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d75bb-790d-4d51-b1e5-b50ff770f836 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:26.931377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d75bb-790d-4d51-b1e5-b50ff770f836 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:26.932841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d75bb-790d-4d51-b1e5-b50ff770f836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:26.934539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d75bb-790d-4d51-b1e5-b50ff770f836 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:26.937045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 22/45] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:26.942901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:27.168856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d46d-5f88-4a8c-a3ea-74dd95ee88b4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:27.204375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a5e5f-0a45-4954-a583-2c64ec8c938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:27.205058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5e5f-0a45-4954-a583-2c64ec8c938a None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:27.205351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5e5f-0a45-4954-a583-2c64ec8c938a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:27.205630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a5e5f-0a45-4954-a583-2c64ec8c938a None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:27.206297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a5e5f-0a45-4954-a583-2c64ec8c938a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:27.207763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a5e5f-0a45-4954-a583-2c64ec8c938a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:27.237527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 24/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:44:26 2022] POST /identity/v3/projects => generated 312 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:28.585582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b36462-604d-4263-ad64-9960bcfd06c0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:28.586123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b36462-604d-4263-ad64-9960bcfd06c0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:28.586213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b36462-604d-4263-ad64-9960bcfd06c0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:28.587534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 23/47] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:28.592999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:28.593504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:28.593913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:28.614629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:28.615823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:28.636345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:28.683096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 25/48] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:28.962965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b36462-604d-4263-ad64-9960bcfd06c0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:29.064535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b36462-604d-4263-ad64-9960bcfd06c0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:29.099570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:29.100206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.100516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.100790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.111760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:29.116607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:29.117406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:29.118128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 24/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:29.123462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:29.220074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09090e84-c5d2-44e9-876a-6e86c0f0442f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:29.254364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:29.254977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.255258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.255532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.256191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:29.256521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:29.257912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4a244-1fd4-43ff-a303-e6a7e60c5281 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:29.266717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 26/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:29.272453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:29.331807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:29.375894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c770c4-4293-4e5b-b527-56d47ac37618 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:29.401390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a875-c93c-4aa1-8350-007680bea863 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.401686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a875-c93c-4aa1-8350-007680bea863 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.401973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73a875-c93c-4aa1-8350-007680bea863 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.403471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 27/51] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:29.408925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.409215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.409495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.410481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:29.411117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.411401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.411676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.412807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:29.414244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:29.415977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 25/52] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:29 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:29.421582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:29.427784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:29.429030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:29.442128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:29.495777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 28/53] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:29.522625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f918-d0b8-4089-8a0a-fe753566368b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:29.544557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:29.555678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:29.556339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.556625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.556898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.557999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:29.559438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:29.561014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 26/54] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:29.567411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:29.664768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47142cde-adde-485c-b445-6c7e39ef99fd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:29.699031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:29.699696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:29.699974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:29.700242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:29.700990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.domain.id=default, user.id=07cb4dfbc4ad477d912360d9e55fab21) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:29.701292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:29.703638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:29.709855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 29/55] 10.222.0.57 () {56 vars in 1322 bytes} [Thu Sep 15 12:44:29 2022] GET /identity/v3/role_assignments?user.id=07cb4dfbc4ad477d912360d9e55fab21&scope.domain.id=default&role.id=13d4313b2fe74b3688999f3077afce5d => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:29.764945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:29.974933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fc6ea-70c0-46eb-aa51-905a53d6fafe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:30.007974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ed7b-dfda-4343-bc1e-04f257b7c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:30.008631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ed7b-dfda-4343-bc1e-04f257b7c693 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:30.008919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ed7b-dfda-4343-bc1e-04f257b7c693 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:30.009197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ed7b-dfda-4343-bc1e-04f257b7c693 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:30.010299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428ed7b-dfda-4343-bc1e-04f257b7c693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:30.011989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428ed7b-dfda-4343-bc1e-04f257b7c693 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:30.013580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 27/56] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:30.019907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:30.035896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:30.118943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fc393-477c-4b47-af09-263d76056d14 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:30.151398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec369-b016-4139-a0c5-ed30c67cc5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:30.152004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec369-b016-4139-a0c5-ed30c67cc5c5 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:30.152309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec369-b016-4139-a0c5-ed30c67cc5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:30.152589 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec369-b016-4139-a0c5-ed30c67cc5c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:30.153242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec369-b016-4139-a0c5-ed30c67cc5c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:30.154685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec369-b016-4139-a0c5-ed30c67cc5c5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:30.180409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 30/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu Sep 15 12:44:30 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:32.343610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605456e5-d611-459f-ab3e-039f2e1f6418 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:32.344138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605456e5-d611-459f-ab3e-039f2e1f6418 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:32.344138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605456e5-d611-459f-ab3e-039f2e1f6418 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:32.345511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 28/58] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:32.350667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:32.351068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:32.351152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:32.368886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:32.370093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:32.426987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:32.436022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 31/59] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:32.704916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605456e5-d611-459f-ab3e-039f2e1f6418 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:32.763626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:32.803732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605456e5-d611-459f-ab3e-039f2e1f6418 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:32.836470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d317-1eec-4d36-9a18-a983525372d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:32.837093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d317-1eec-4d36-9a18-a983525372d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:32.837380 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d317-1eec-4d36-9a18-a983525372d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:32.837658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d317-1eec-4d36-9a18-a983525372d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:32.838769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47d317-1eec-4d36-9a18-a983525372d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:32.840209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47d317-1eec-4d36-9a18-a983525372d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:32.841788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 29/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:32.847523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:32.943625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5116b-95f0-48f1-829c-b5d17f481bf4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:32.975481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c55a-8701-4a87-b689-19a365056739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:32.976017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c55a-8701-4a87-b689-19a365056739 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:32.976309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c55a-8701-4a87-b689-19a365056739 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:32.976584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c55a-8701-4a87-b689-19a365056739 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:32.977224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c55a-8701-4a87-b689-19a365056739 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:32.978642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c55a-8701-4a87-b689-19a365056739 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:33.004949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 32/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:44:32 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:36.265173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd617e8-c710-4fc6-834e-ecdf815886a1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.266169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd617e8-c710-4fc6-834e-ecdf815886a1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.266681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd617e8-c710-4fc6-834e-ecdf815886a1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.268284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 30/62] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:36.273898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeef0-a277-4682-a0b9-bf7d4407e621 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.274302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeef0-a277-4682-a0b9-bf7d4407e621 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.274466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeef0-a277-4682-a0b9-bf7d4407e621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.313318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-29afeef0-a277-4682-a0b9-bf7d4407e621 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:36.314549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-29afeef0-a277-4682-a0b9-bf7d4407e621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:36.379640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:36.410268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 33/63] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:36.463549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ca7bf-2f84-4ad2-bbb2-952c4e823a1a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.464185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ca7bf-2f84-4ad2-bbb2-952c4e823a1a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.464686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ca7bf-2f84-4ad2-bbb2-952c4e823a1a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.466352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 31/64] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:36.472514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146fd0d-456b-4221-96f8-f01f41399331 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.472810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146fd0d-456b-4221-96f8-f01f41399331 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.473091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146fd0d-456b-4221-96f8-f01f41399331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.501349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7146fd0d-456b-4221-96f8-f01f41399331 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:36.502574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7146fd0d-456b-4221-96f8-f01f41399331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:36.601635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 34/65] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:36.751235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ca7bf-2f84-4ad2-bbb2-952c4e823a1a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:36.820795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a426-3dd6-4cc5-9375-82d15a1c8a1f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.821345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a426-3dd6-4cc5-9375-82d15a1c8a1f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.821862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8a426-3dd6-4cc5-9375-82d15a1c8a1f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.823928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 35/66] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:36.830490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5155-2bc6-4565-b5b0-67efa48d1667 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.831003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5155-2bc6-4565-b5b0-67efa48d1667 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.831515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5155-2bc6-4565-b5b0-67efa48d1667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.853703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-565e5155-2bc6-4565-b5b0-67efa48d1667 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:36.855319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-565e5155-2bc6-4565-b5b0-67efa48d1667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:36.872862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3ca7bf-2f84-4ad2-bbb2-952c4e823a1a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:36.903309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:36.916770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:36.918017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.918480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.918932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.934049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:36.934438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 36/67] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:36.936315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a319b-5f75-4681-a707-18de02eae269 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:36.939520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:36.940358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:36.941096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:36.941661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a319b-5f75-4681-a707-18de02eae269 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.941661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a319b-5f75-4681-a707-18de02eae269 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.942900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 37/68] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:36.943770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 32/69] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:36.953558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad3228-02e1-4bd8-97d4-25368438c3bb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:36.953705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:36.953847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad3228-02e1-4bd8-97d4-25368438c3bb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:36.954336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad3228-02e1-4bd8-97d4-25368438c3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:36.973617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-43ad3228-02e1-4bd8-97d4-25368438c3bb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:36.975126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-43ad3228-02e1-4bd8-97d4-25368438c3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:37.025260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.062617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 38/70] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:37.064073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7819b8-8c61-46c0-aa3a-66423a987e18 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.064512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7819b8-8c61-46c0-aa3a-66423a987e18 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.065503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7819b8-8c61-46c0-aa3a-66423a987e18 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.067154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 39/71] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:37.068128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7819b8-8c61-46c0-aa3a-66423a987e18 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.078987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590c482-7f22-4d22-8ae7-0bd2e2528d05 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.119133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.120047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.120560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.121031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.121998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:37.122545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:37.125184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed2768-c036-4f71-9526-0dc38249ccdb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:37.138477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 33/72] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:37.140911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.141398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.141938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.161332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:37.162865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:37.190684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7819b8-8c61-46c0-aa3a-66423a987e18 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.216806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.227264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.228123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.228580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.228995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.239899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:37.241744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 34/73] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:37.242579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:37.245574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:37.246587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:37.247470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:37.250078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 40/74] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:37.251134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.283774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.353803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1ae6c-ce22-4054-a1a7-d9cd8641a29d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.355879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1449d-cb9b-4fec-893f-55cf30209f0f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.390206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.390845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.391128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.391441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.392596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.393354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.393642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.394039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.405822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:37.408567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:37.410017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:37.410958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:37.411751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:37.412548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:37.413105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 35/75] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:37.413105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.414254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 41/76] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:37.415606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.457258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.512095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.523927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3e88f-9db0-4f7b-ab85-ec7294616888 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.525495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017959-43a3-498c-aace-c56fe47bc715 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.558061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.558623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.558904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.559177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.559920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.560262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:37.560566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.560880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.561177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.561826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:37.562059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:37.562147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:37.563474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:37.563645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 42/77] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:37.564432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.579186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 36/78] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:37.580016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.665779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e46cf-dbfc-4240-8e37-e3701adf21e7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.676816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.680087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14051bb6-77d8-493f-9bd3-30b4166837a1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.699759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.700426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.700715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.700994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.713650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.714263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.714549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.714832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.715494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:37.715802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:37.717768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:37.725537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 37/79] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:37.726326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.732251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.774124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:37.775608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:37.827512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342ea0d-25d2-4251-a159-c1ada97ba24b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.860061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:37.860725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:37.861011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:37.861288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:37.870813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 43/80] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:37.871445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:37.871711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:37.877834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:37.878795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 38/81] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:37.881003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:37.906821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.931668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:37.974231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a94f1-0da6-4d4f-8aef-16dd7741e669 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:37.983610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff9a1e-0245-475f-b3f5-d8e00eeff308 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.008100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.008746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.009027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.009492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.009949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.011652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.017734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.018362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.018745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.018922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.020050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.021522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.023096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 39/82] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:37 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:38.025087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.048367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 44/83] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:44:37 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:38.048367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.152762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:38.165230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18836d31-63ef-49d6-b14b-80202cc0a10e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.189544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71735b3-c34a-4ef9-b4ef-5630964d14f4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.207071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.208123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.208624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.208924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:38.209300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.210701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.212389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.214188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 40/84] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:38.215133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.238598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.239256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.239545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.240063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.241015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.242478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.244166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 45/85] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:38.245063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.371097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f0e2d-8d6b-411f-8604-abd7476b72bd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.409001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d45be3-5cdf-45d5-8a1a-80fe51077596 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.426917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.427585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.427878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.428152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.432999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:38.433441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.434966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.448139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 41/86] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:38.452293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.455355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.456252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.456789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.457261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.478536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:38.536735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:38.556327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.558085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.596508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95135a0e-f1d4-4b66-b587-9cefb97105be None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.653020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.653714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.654425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.655428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.689618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 46/87] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:38.690838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.749601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.751350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.781088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:38.792041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:38.813488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae5659-0002-4ab4-826b-b157ec1b8367 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:38.856829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:38.858037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:38.858403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:38.858757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:38.859911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=2921a61a62e3462084377c936b42cb02) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:38.860312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:38.862529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:38.868757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 42/88] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:38.876599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 47/89] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/role_assignments?user.id=2921a61a62e3462084377c936b42cb02&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:38.876599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.878039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:38.999743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f541fa70-1f94-4c9e-aac0-1c6f9e0705a1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.041739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.042640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.043003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.043355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.044905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.048080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.050066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 48/90] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:39.050865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:39.071167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:39.130615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0b8d4-15d8-4568-9fda-af137130cc22 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.184024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:39.189046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f88e3-4ef0-439d-9171-f9ae3899d3ab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.216171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.217016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.217969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.219771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.221742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.222841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.224932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 43/91] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:38 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:39.225612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:39.236933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.237854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.238282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.238684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.240481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.242855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.245897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 49/92] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:39 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:39.246905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:39.350574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:39.373063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1aea5-e318-4d32-b8e7-49e3415a2be9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.417060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.417727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70f9a0-d98a-4c3b-9c4e-733b27e5f187 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.418792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.419473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.419889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.421370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=07cb4dfbc4ad477d912360d9e55fab21) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:39.421813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.423405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.432491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 50/93] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:39 2022] GET /identity/v3/role_assignments?user.id=07cb4dfbc4ad477d912360d9e55fab21&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:39.433581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:39.505694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.506286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.506617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.506899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.566126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842fc96-1026-4a12-9252-cba92a40e4ea None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.608687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.610110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.610610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.611146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.612380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] RBAC: Adding query filter params (role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=2921a61a62e3462084377c936b42cb02) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:39.612947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.615377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.622856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 51/94] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:39 2022] GET /identity/v3/role_assignments?user.id=2921a61a62e3462084377c936b42cb02&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=0f55ac654fb247249b38e3c701826f02 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:39.631858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:39.660697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.662213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.794529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 44/95] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:39.801019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:39.815957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:39.917758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd602c0-9c88-4ce1-9619-535e98dc0f63 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:39.965362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd3df6-f030-443f-8ffa-f724de95f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:39.966022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd3df6-f030-443f-8ffa-f724de95f7fd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:39.966305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd3df6-f030-443f-8ffa-f724de95f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:39.966586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd3df6-f030-443f-8ffa-f724de95f7fd None admin] PATH_INFO: `/v3/projects/f2692caece4344a6b60f0019f18d1df0` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:39.972975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd3df6-f030-443f-8ffa-f724de95f7fd None admin] RBAC: Authorizing `identity:get_project(project_id=f2692caece4344a6b60f0019f18d1df0)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:39.974455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd3df6-f030-443f-8ffa-f724de95f7fd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:39.976420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 52/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:39 2022] GET /identity/v3/projects/f2692caece4344a6b60f0019f18d1df0 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:39.982445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:40.103613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:40.127077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97711a15-207d-4c90-84d0-031d379bfd9b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:40.189200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:40.189741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:40.190021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:40.190295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:40.191022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=f2692caece4344a6b60f0019f18d1df0, user.id=2921a61a62e3462084377c936b42cb02) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:40.191333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:40.192803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472678a9-af12-41e5-9cf9-891c060584e7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:40.207454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 45/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:39 2022] GET /identity/v3/role_assignments?user.id=2921a61a62e3462084377c936b42cb02&scope.project.id=f2692caece4344a6b60f0019f18d1df0&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:41.024300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ff300-19aa-486c-9238-af248d4b9363 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:41.024833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ff300-19aa-486c-9238-af248d4b9363 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:41.024833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ff300-19aa-486c-9238-af248d4b9363 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:41.026310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 53/98] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:41.032708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc0302-2c83-439a-ae9f-3a3c94565cc3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:41.033604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc0302-2c83-439a-ae9f-3a3c94565cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:41.034095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc0302-2c83-439a-ae9f-3a3c94565cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:41.062048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-62cc0302-2c83-439a-ae9f-3a3c94565cc3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:41.063684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-62cc0302-2c83-439a-ae9f-3a3c94565cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:41.151963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 46/99] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:41.206684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922a9df-efb6-4a67-b728-e73174aef504 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:41.207289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922a9df-efb6-4a67-b728-e73174aef504 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:41.207757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922a9df-efb6-4a67-b728-e73174aef504 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:41.209669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 54/100] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:41.222413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:41.222958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:41.223448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:41.245278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:41.246460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:41.336117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 47/101] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:41.628709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922a9df-efb6-4a67-b728-e73174aef504 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:41.743948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:41.773875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922a9df-efb6-4a67-b728-e73174aef504 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:41.822412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:41.823387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:41.823859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:41.824392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:41.825728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:41.827463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:41.829595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 55/102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:41.835463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:41.855595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:41.907826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:41.972160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0a70b-09bf-4f39-ba9a-ee6cf4596d8b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:42.026365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7ce35-6599-44e5-8719-9058bf21f805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:42.027359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7ce35-6599-44e5-8719-9058bf21f805 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.027860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7ce35-6599-44e5-8719-9058bf21f805 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.028355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7ce35-6599-44e5-8719-9058bf21f805 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.029307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7ce35-6599-44e5-8719-9058bf21f805 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:42.030992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc7ce35-6599-44e5-8719-9058bf21f805 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:42.075550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 48/103] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:44:41 2022] POST /identity/v3/projects => generated 302 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:44:42.132180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedecdf8-6990-4a91-825d-a031bfc77400 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:42.186085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:42.186719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.187003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.187278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.190333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248a086-77d3-4ce9-b204-ec2076d50398 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.190796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248a086-77d3-4ce9-b204-ec2076d50398 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.191210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248a086-77d3-4ce9-b204-ec2076d50398 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.196151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 49/104] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:42.198757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:42.201834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a80fa-1b92-4d77-bf22-c323aa57241c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.202549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a80fa-1b92-4d77-bf22-c323aa57241c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.203118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a80fa-1b92-4d77-bf22-c323aa57241c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:42.205023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:42.205850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:42.206582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 56/105] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:44:42.213248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:42.240674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:42.255045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bc5a80fa-1b92-4d77-bf22-c323aa57241c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:42.256526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bc5a80fa-1b92-4d77-bf22-c323aa57241c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:42.365582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:42.366810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 50/106] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:42.427193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8d0a2-5de4-45f8-a2f4-88b8066bfb80 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:42.467675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d2c8-3c8d-465d-a318-0d29259cccfb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.467964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d2c8-3c8d-465d-a318-0d29259cccfb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.468553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67d2c8-3c8d-465d-a318-0d29259cccfb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.470354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 51/107] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:42.475532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.476057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.476630 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.477848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:42.478472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.478761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.479041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.479720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:42.480031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:42.481448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:42.491443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 57/108] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:42.509439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:42.509719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:42.511486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:42.594120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 52/109] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 15 12:44:42.633991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:42.683071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13058400-aa9b-4f55-81b1-2aa9332021e6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:42.702092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:42.733335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:42.733947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.734234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.734539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] PATH_INFO: `/v3/users/bc4322dd0c57429bad559e37d5e75137` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.754002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4322dd0c57429bad559e37d5e75137)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:42.755365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:42.757388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 58/110] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:42 2022] GET /identity/v3/users/bc4322dd0c57429bad559e37d5e75137 => generated 289 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:42.764950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:42.847588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d28264-9223-441a-be37-4f43bab25ec4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:42.878413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:42.891509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:42.892479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:42.893228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:42.893709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:42.908486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:42.917452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:42.918294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:42.919030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:42.922120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 53/111] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:44:42.927919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:42.969885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab14d0-922c-451c-9b82-7f3fd5247172 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.003902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.033197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2be05-5757-47e4-abdc-ee7a81e40d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.038929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2be05-5757-47e4-abdc-ee7a81e40d82 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.039642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2be05-5757-47e4-abdc-ee7a81e40d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.040117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2be05-5757-47e4-abdc-ee7a81e40d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.098867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4a06f-817a-4a32-bf35-b8e2972fd8c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.152539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2be05-5757-47e4-abdc-ee7a81e40d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.154228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2be05-5757-47e4-abdc-ee7a81e40d82 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.180589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.181215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.181501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.181772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.182431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:43.182736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.184110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.193538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 54/112] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:43.194140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.241970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.269453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 59/113] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:44:43.275773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49597d6-c169-4490-a344-502908ae32e8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.276067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49597d6-c169-4490-a344-502908ae32e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.276367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49597d6-c169-4490-a344-502908ae32e8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.277842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 60/114] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:43.278565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49597d6-c169-4490-a344-502908ae32e8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.306101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d340c-211e-4d72-b0b1-8631375ad11a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.309208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.343950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.344693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.344978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.345255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.356210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:43.362316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:43.363163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:43.365492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 55/115] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:43.372315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.392875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49597d6-c169-4490-a344-502908ae32e8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.431953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fa288-26a4-4190-9c6c-611a6691ba37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.432743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa288-26a4-4190-9c6c-611a6691ba37 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.433120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa288-26a4-4190-9c6c-611a6691ba37 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.433811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fa288-26a4-4190-9c6c-611a6691ba37 None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.435042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa288-26a4-4190-9c6c-611a6691ba37 None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.437967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fa288-26a4-4190-9c6c-611a6691ba37 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.439969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 61/116] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:43.442113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.442463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.442837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.469043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:43.470941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:43.502390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cf2de-8f59-462a-8a14-77fd68e004b2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.511723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.544494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.545815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.546186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.546619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.547791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.549372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.549961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 62/117] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:43.551086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.556252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 56/118] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:43.557219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.578626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.661374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fa69f-d3d5-4dee-94bd-a05d23f4ad89 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.683647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e8748-53ef-432c-a2ec-d9c9f522fb39 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.697999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.698844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.699297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.699734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.700599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:43.701095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.703296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.711632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 63/119] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:43.716907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.722189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd6457-7bbd-4083-9efd-f440162b8373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.722840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd6457-7bbd-4083-9efd-f440162b8373 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.723194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd6457-7bbd-4083-9efd-f440162b8373 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.723543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd6457-7bbd-4083-9efd-f440162b8373 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.724825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd6457-7bbd-4083-9efd-f440162b8373 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.726384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd6457-7bbd-4083-9efd-f440162b8373 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.728694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 57/120] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:43.729476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc28620-c1cf-491c-8fdf-e0f67b8a4296 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.729834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc28620-c1cf-491c-8fdf-e0f67b8a4296 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.730220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc28620-c1cf-491c-8fdf-e0f67b8a4296 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.731634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 58/121] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:43.732437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc28620-c1cf-491c-8fdf-e0f67b8a4296 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.783453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.809557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:43.826132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0920cc-ff82-49da-96e2-5207bdc3b260 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.851770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc28620-c1cf-491c-8fdf-e0f67b8a4296 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:43.861223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.861833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.862114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.862387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.863109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] RBAC: Adding query filter params (role.id=6f21288e54a24472b723fe127cae383e, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=bc4322dd0c57429bad559e37d5e75137) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:43.863416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.864933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.870319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 64/122] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/role_assignments?user.id=bc4322dd0c57429bad559e37d5e75137&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=6f21288e54a24472b723fe127cae383e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:43.875161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:43.891330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14042b4d-3262-436a-b2e9-acd2f1e0e87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:43.891992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14042b4d-3262-436a-b2e9-acd2f1e0e87b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.892301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14042b4d-3262-436a-b2e9-acd2f1e0e87b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.892589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14042b4d-3262-436a-b2e9-acd2f1e0e87b None admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.893794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14042b4d-3262-436a-b2e9-acd2f1e0e87b None admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:43.895560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14042b4d-3262-436a-b2e9-acd2f1e0e87b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:43.897272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 59/123] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:43 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:43.898507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:43.898795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:43.899071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:43.919607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:43.921039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:44.003945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 60/124] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:44.005181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.009354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bcf15-a6e3-4181-b1cd-2b964abd1f74 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.085321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:44.090627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.092955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.093618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.094117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/2921a61a62e3462084377c936b42cb02/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.096414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] RBAC: Authorizing `identity:create_grant(project_id=b59720c712524d768e1fc85c3cf651b5, user_id=2921a61a62e3462084377c936b42cb02, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:44.098290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:44.117235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 65/125] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:43 2022] PUT /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/2921a61a62e3462084377c936b42cb02/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:44.118097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.135248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:44.225562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b125c7b-42ad-4f2f-a88e-08ea05137100 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.335937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.336536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.336845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.337122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.354794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:44.360757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:44.361585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:44.362347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 61/126] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:44.362954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.375412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1106c90-4089-4ba9-925c-f82c52ad9d0c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.427508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.428164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.429763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.430055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.432149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:44.434131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:44.437252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 66/127] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:44.440266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.535128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:44.539585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:44.539972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f6411-d94c-4897-8a44-bea3aaf38e61 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.565355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002802d6-0a73-496f-a2fb-bada71aaba1b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.607873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.608731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.609024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.609324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.610135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:44.610833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:44.612507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:44.620892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 67/128] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:44.621817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.622377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.623924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.624446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.624925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.649101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:44.660063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:44.660944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:44.661690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:44.664379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 62/129] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:44.670747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.819511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e3ff1-1519-4650-bade-ea5de7ba1337 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.867426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acbb0b-38fc-41d9-b7d4-517e78276994 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:44.886300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.886979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.887267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.887549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/07cb4dfbc4ad477d912360d9e55fab21/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.891249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] RBAC: Authorizing `identity:create_grant(project_id=b59720c712524d768e1fc85c3cf651b5, user_id=07cb4dfbc4ad477d912360d9e55fab21, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:44.893968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:44.918576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 68/130] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:44 2022] PUT /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/07cb4dfbc4ad477d912360d9e55fab21/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:44.919515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:44.924651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:44.948027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:44.948724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:44.949012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:44.949293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:44.952116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:44.954603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:44.959385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 63/131] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:44.963631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:45.015641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:45.114244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916f85b-2209-4dfa-94f0-eb1aaa0d5d29 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:45.156660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:45.157795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.158322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.158752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.160199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:45.161874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:45.164342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 64/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:45.169848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:45.189676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4fbb-77c6-43da-ad1f-92aa32f6ec47 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:45.265299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:45.265774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.266059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.266344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.267022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:45.267330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:45.269092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956705b-5fee-47cb-92f3-c3873ded3b47 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:45.278109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 69/133] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:45.279343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8ece-fa1f-4f82-b97f-91ab3cfea538 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.279618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8ece-fa1f-4f82-b97f-91ab3cfea538 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.280015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8ece-fa1f-4f82-b97f-91ab3cfea538 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.286944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 70/134] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:45.290392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8ece-fa1f-4f82-b97f-91ab3cfea538 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:45.321002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:45.329648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4a940-d89e-46cd-8fd4-53eea3957666 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:45.367382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:45.375645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ee661-8443-403c-8132-263cae52dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:45.376592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ee661-8443-403c-8132-263cae52dfc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.377026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ee661-8443-403c-8132-263cae52dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.377476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ee661-8443-403c-8132-263cae52dfc6 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/2921a61a62e3462084377c936b42cb02/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.379847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ee661-8443-403c-8132-263cae52dfc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b59720c712524d768e1fc85c3cf651b5, user_id=2921a61a62e3462084377c936b42cb02, role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:45.381539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ee661-8443-403c-8132-263cae52dfc6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:45.398916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 65/135] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:45 2022] PUT /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/2921a61a62e3462084377c936b42cb02/roles/0f55ac654fb247249b38e3c701826f02 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:45.400923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.401541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.402003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.433797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8ece-fa1f-4f82-b97f-91ab3cfea538 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:45.447210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:45.448773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:45.511247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:45.512223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.512695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.513403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.516540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:45.518517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:45.525167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 71/136] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:45 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:45.531193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:45.598820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 66/137] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:45.680379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f631f1-c69e-4ffa-8861-ce0825d0a1d9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:45.712427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:45.720219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:45.721104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.721572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.722008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] PATH_INFO: `/v3/projects/f2692caece4344a6b60f0019f18d1df0` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.723298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] RBAC: Authorizing `identity:get_project(project_id=f2692caece4344a6b60f0019f18d1df0)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:45.724992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:45.728248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 72/138] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:45 2022] GET /identity/v3/projects/f2692caece4344a6b60f0019f18d1df0 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:45.733826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:45.833008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:45.854876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa927d5-7707-4cb3-b6f2-1e3a6b4323e7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:45.902347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:45.903037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:45.903325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:45.903665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] PATH_INFO: `/v3/projects/f2692caece4344a6b60f0019f18d1df0/users/2921a61a62e3462084377c936b42cb02/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:45.905881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] RBAC: Authorizing `identity:create_grant(project_id=f2692caece4344a6b60f0019f18d1df0, user_id=2921a61a62e3462084377c936b42cb02, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:45.907345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:45.927018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 67/139] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:45 2022] PUT /identity/v3/projects/f2692caece4344a6b60f0019f18d1df0/users/2921a61a62e3462084377c936b42cb02/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:46.077069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:46.214924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c2d24-9238-473f-986a-771cb34a2278 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:46.260751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ca058-9507-49c7-9769-d23b8dca3242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:46.261991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ca058-9507-49c7-9769-d23b8dca3242 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:46.262466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ca058-9507-49c7-9769-d23b8dca3242 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:46.262958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ca058-9507-49c7-9769-d23b8dca3242 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:46.264842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ca058-9507-49c7-9769-d23b8dca3242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:46.266490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ca058-9507-49c7-9769-d23b8dca3242 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:46.268603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 73/140] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:46.273893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:46.417590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68178978-608a-4b73-a7ff-c92d9fa3a9cf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:46.456984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b3dbe-4e44-4497-b162-7d686dea9b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:46.457611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b3dbe-4e44-4497-b162-7d686dea9b27 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:46.457898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b3dbe-4e44-4497-b162-7d686dea9b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:46.458189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b3dbe-4e44-4497-b162-7d686dea9b27 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:46.458896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b3dbe-4e44-4497-b162-7d686dea9b27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:46.460440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b3dbe-4e44-4497-b162-7d686dea9b27 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:46.495006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 68/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:44:46 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:47.018002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481cefc-2227-4e57-9b7d-40bfa08c4b3e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:47.018002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481cefc-2227-4e57-9b7d-40bfa08c4b3e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:47.018002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481cefc-2227-4e57-9b7d-40bfa08c4b3e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:47.018002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 74/142] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:47.023589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051ab62-526b-454e-9ea0-c9605653b445 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:47.023891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051ab62-526b-454e-9ea0-c9605653b445 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:47.024181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051ab62-526b-454e-9ea0-c9605653b445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:47.034796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:47.049812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4051ab62-526b-454e-9ea0-c9605653b445 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:47.051016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4051ab62-526b-454e-9ea0-c9605653b445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:47.147965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 69/143] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:47.537188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eecbd9-117d-4796-835f-20dc01dc98af None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:47.537572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eecbd9-117d-4796-835f-20dc01dc98af None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:47.537746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eecbd9-117d-4796-835f-20dc01dc98af None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:47.539415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 75/144] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:47.549661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:47.550105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:47.550281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:47.574352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:47.575612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:47.605519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eecbd9-117d-4796-835f-20dc01dc98af None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:47.694552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 70/145] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:47.722376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:47.776511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eecbd9-117d-4796-835f-20dc01dc98af None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:47.823423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8c65a-c10c-4b6b-9af2-4631125d6e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:47.823979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8c65a-c10c-4b6b-9af2-4631125d6e6e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:47.824256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8c65a-c10c-4b6b-9af2-4631125d6e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:47.824621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8c65a-c10c-4b6b-9af2-4631125d6e6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:47.841385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8c65a-c10c-4b6b-9af2-4631125d6e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d7a8c65a-c10c-4b6b-9af2-4631125d6e6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:47.847710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:47.848580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:47.849327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 76/146] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:44:47.854834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:47.927075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:47.974836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da9863-223e-4a2c-9b77-eae1437c209f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:48.019834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:48.020484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.020771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.021046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.021717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:48.022017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:48.023413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:48.033824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4c237-7a6e-4549-85cf-12731d50ee0c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.034321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4c237-7a6e-4549-85cf-12731d50ee0c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.034395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4c237-7a6e-4549-85cf-12731d50ee0c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.035711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 77/147] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:48.039071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 71/148] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:48.041005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.041285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.041558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.069387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:48.080523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:48.081658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:48.212495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 78/149] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:48.213889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:48.257448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59267f-7a18-48e9-8823-77bf5f1d86f9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:48.288837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:48.308683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:48.309341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.309633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.309927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] PATH_INFO: `/v3/users/bc4322dd0c57429bad559e37d5e75137` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.311984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] RBAC: Authorizing `identity:get_user(user_id=bc4322dd0c57429bad559e37d5e75137)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:48.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:48.315321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 72/150] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:48 2022] GET /identity/v3/users/bc4322dd0c57429bad559e37d5e75137 => generated 289 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:48.322192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:48.325909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:48.401143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b1216-6fec-4adc-865e-85d0bb3c44c3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:48.464791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:48.465478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.465760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.466035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.489351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:48.494437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:48.495413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:48.496628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 79/151] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:48.512993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f19a3f-8d4d-47b5-b974-de4cabdd88db None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:48.515016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:48.558739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:48.559453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.559649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.559923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.561509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:48.562984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:48.564669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 73/152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:48 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:48.570565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:48.677960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:48.724253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf001-ee86-47a1-98e8-998c91588413 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:48.740012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04734286-42eb-4b75-b0fa-018490a1e6b7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:48.751340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:48.792986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7be998-345b-4c4d-8f2f-4a78fa4216b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:48.793673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be998-345b-4c4d-8f2f-4a78fa4216b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.793955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be998-345b-4c4d-8f2f-4a78fa4216b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.794229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7be998-345b-4c4d-8f2f-4a78fa4216b8 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/bc4322dd0c57429bad559e37d5e75137/roles/6f21288e54a24472b723fe127cae383e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.796957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:48.797327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7be998-345b-4c4d-8f2f-4a78fa4216b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b59720c712524d768e1fc85c3cf651b5, user_id=bc4322dd0c57429bad559e37d5e75137, role_id=6f21288e54a24472b723fe127cae383e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:48.797587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.797872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.798150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.798782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7be998-345b-4c4d-8f2f-4a78fa4216b8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:48.798895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:48.799122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:48.800573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692c913-dd30-4685-8e2e-5339b765542e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:48.814953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 80/153] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:48.816225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e933cb-1178-4c25-b32f-015ae67f9dba None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.816540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e933cb-1178-4c25-b32f-015ae67f9dba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.816817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e933cb-1178-4c25-b32f-015ae67f9dba None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.818184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 81/154] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:48.821763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 74/155] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:48 2022] PUT /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5/users/bc4322dd0c57429bad559e37d5e75137/roles/6f21288e54a24472b723fe127cae383e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:48.825324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e933cb-1178-4c25-b32f-015ae67f9dba None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:48.829823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:48.830135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:48.830379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:48.865530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:48.866807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:48.990791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 75/156] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:48.992262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.054082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e933cb-1178-4c25-b32f-015ae67f9dba None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:49.085567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:49.137661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:49.138318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.139235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.139657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.140900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:49.142385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:49.144960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 82/157] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:48 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:49.149696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.186579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67d968-7a0f-4783-9836-c1966f7c8655 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:49.239290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:49.245141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:49.246093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.246768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.247318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.269203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:49.274988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:49.275920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:49.276721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:49.279069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 76/158] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:49.285244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.328753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70ca84-a524-4561-9f3e-e82a4a7711ee None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:49.387380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:49.387995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.388288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.389115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.471373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ab6a9-ac32-4a98-89b8-e0fbd324bdd8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:49.492107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:49.500577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:49.501985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:49.534777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:49.535806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.536352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.536840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.537784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:49.538300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:49.539960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:49.555578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 77/159] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:49.561430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.583302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:49.663926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 83/160] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:44:49.664828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.733597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd5f9-46a6-45e0-b134-edf39b405bd7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:49.807482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:49.808046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.808347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.808627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] PATH_INFO: `/v3/users/07cb4dfbc4ad477d912360d9e55fab21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.813023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] RBAC: Authorizing `identity:get_user(user_id=07cb4dfbc4ad477d912360d9e55fab21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:49.814546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:49.816405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 78/161] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:49 2022] GET /identity/v3/users/07cb4dfbc4ad477d912360d9e55fab21 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:49.816957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.837800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848c1cb-b68a-4e63-9d46-ef6dc6931880 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:49.902724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:49.907994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45d586-c592-4fa4-bbcc-a7bb5cacb21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:49.909042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45d586-c592-4fa4-bbcc-a7bb5cacb21e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.910780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45d586-c592-4fa4-bbcc-a7bb5cacb21e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.911209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45d586-c592-4fa4-bbcc-a7bb5cacb21e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.933439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45d586-c592-4fa4-bbcc-a7bb5cacb21e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3b45d586-c592-4fa4-bbcc-a7bb5cacb21e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:49.942304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:49.943464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:49.944447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:49.949031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 84/162] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:44:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:49.952182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529cfa-f05a-46c9-80a1-bd9e6579f8f8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:49.956084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529cfa-f05a-46c9-80a1-bd9e6579f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:49.957653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529cfa-f05a-46c9-80a1-bd9e6579f8f8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:49.959272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 85/163] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:49.961363 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529cfa-f05a-46c9-80a1-bd9e6579f8f8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:49.982167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:50.030228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066c458-741d-432a-8837-7ec7d6bc710c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:50.112054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:50.112730 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.113008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.113280 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.115397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:50.117171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:50.118818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 79/164] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:49 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:50.119582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:50.186476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529cfa-f05a-46c9-80a1-bd9e6579f8f8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:50.253597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:50.254211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.254493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.254773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.316321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1093657-5bcc-4172-bb05-d312eff3a53e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:50.337615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:50.376156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:50.377601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:50.395536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:50.406801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:50.407419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.407703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.407977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.408664 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:50.408977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:50.410353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1b7ee-f589-4e8f-9891-a8fc2777842f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:50.435058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 80/165] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:44:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:50.437945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.438240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.438522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.487090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:50.488337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:50.606828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 86/166] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:50.608296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:50.632469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 81/167] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:50.634005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:50.748487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:50.753276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835dfa9-cc4a-478c-a4b6-e5259a5e0db4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:50.790715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:50.791789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.792264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.792981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.794571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:50.796566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:50.798693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 82/168] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:50 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:50.799664 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:50.843440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156107a-fe9e-44de-b080-b29979aef55b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:50.851120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:50.918352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:50.918877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.919154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.919424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.921035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=2921a61a62e3462084377c936b42cb02) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:50.921444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:50.922707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a31d2-1fb5-4198-9935-540bda4825a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:50.922886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:50.934510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 87/169] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:50 2022] GET /identity/v3/role_assignments?user.id=2921a61a62e3462084377c936b42cb02&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:50.935276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:50.961192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:50.962488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:50.962997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:50.963474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:50.964930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:50.966785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:50.969282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 83/170] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:50 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:50.994508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:51.058551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:51.090829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55b532-dd35-449f-8a5d-ae3136a60a59 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:51.157135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:51.157777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:51.158055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:51.158323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:51.228703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fcc9-d04f-4c7a-92b9-a56d15ce845e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:51.234771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:51.262643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:51.268440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:51.318473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:51.319105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:51.319388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:51.319656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:51.320424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=07cb4dfbc4ad477d912360d9e55fab21) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:51.320736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:51.322249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:51.333716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 84/171] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:50 2022] GET /identity/v3/role_assignments?user.id=07cb4dfbc4ad477d912360d9e55fab21&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:51.334598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:51.442641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 88/172] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:51.448811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:51.581601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:51.607120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:51.613214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d2dd9-3257-4157-947e-3b9cca19129c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:51.644190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024a312-cf82-40db-81c2-e04ce0c2b780 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:51.670709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a600c25-dc29-4d6b-b146-b8f911060a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:51.672776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a600c25-dc29-4d6b-b146-b8f911060a8d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:51.673370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a600c25-dc29-4d6b-b146-b8f911060a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:51.674939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a600c25-dc29-4d6b-b146-b8f911060a8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:51.691089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a600c25-dc29-4d6b-b146-b8f911060a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8a600c25-dc29-4d6b-b146-b8f911060a8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:51.696715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:51.697734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:51.698708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:51.700646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 85/173] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:51.704386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba92f9-9e85-41ce-ba02-ded993dff412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:51.704386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba92f9-9e85-41ce-ba02-ded993dff412 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:51.704386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba92f9-9e85-41ce-ba02-ded993dff412 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:51.704386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba92f9-9e85-41ce-ba02-ded993dff412 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:51.705217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba92f9-9e85-41ce-ba02-ded993dff412 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:51.706188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469985f-c5ad-4932-8841-d4f2c9749888 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:51.706613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469985f-c5ad-4932-8841-d4f2c9749888 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:51.706721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba92f9-9e85-41ce-ba02-ded993dff412 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:51.707122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469985f-c5ad-4932-8841-d4f2c9749888 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:51.708535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 86/175] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:51.712296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 89/175] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:51 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:51.713459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469985f-c5ad-4932-8841-d4f2c9749888 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:51.715515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:51.715803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:51.716077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:51.739602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:51.740834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:51.875760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 90/176] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:51.876709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:51.919587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469985f-c5ad-4932-8841-d4f2c9749888 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:51.999191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:51.999818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.000099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.000395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.001067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:52.001369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:52.002766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:52.023237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 87/177] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:52.026899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:52.030661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.065477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:52.086507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e78543-082d-4b7b-883d-1f78992c8bab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.140942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:52.141929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.142421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.142892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.143829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] RBAC: Adding query filter params (role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=2921a61a62e3462084377c936b42cb02) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:52.145275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:52.148121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b96d91-104f-49fb-a9f2-c882e7d7f33e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:52.163434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 91/178] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:51 2022] GET /identity/v3/role_assignments?user.id=2921a61a62e3462084377c936b42cb02&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=0f55ac654fb247249b38e3c701826f02 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:52.165122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248702-7236-4b1e-ad11-7ee2a4ad03a3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.166906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248702-7236-4b1e-ad11-7ee2a4ad03a3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.167508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248702-7236-4b1e-ad11-7ee2a4ad03a3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.176592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 92/179] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:52.178424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4335-ba9f-4c9c-b06a-524d5ea7b501 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.180885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4335-ba9f-4c9c-b06a-524d5ea7b501 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.181737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4335-ba9f-4c9c-b06a-524d5ea7b501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.225368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-275d4335-ba9f-4c9c-b06a-524d5ea7b501 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:52.229317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-275d4335-ba9f-4c9c-b06a-524d5ea7b501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:52.253665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81415b63-dd9a-4930-a807-6c61eed31a52 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.322807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:52.323457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.323741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.324016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] PATH_INFO: `/v3/users/2921a61a62e3462084377c936b42cb02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.325336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=2921a61a62e3462084377c936b42cb02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:52.327357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:52.329175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 88/180] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/users/2921a61a62e3462084377c936b42cb02 => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:52.330009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.352473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 93/181] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:52.354498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e9a50-795e-44f0-8a2f-fa9eaa8ff210 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.355018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e9a50-795e-44f0-8a2f-fa9eaa8ff210 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.355525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e9a50-795e-44f0-8a2f-fa9eaa8ff210 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.357239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 94/182] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:52.364347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e9a50-795e-44f0-8a2f-fa9eaa8ff210 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.459186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:52.484606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:52.496438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17aa10-cb6f-4530-b0ef-3526942fc9c2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.533821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e9a50-795e-44f0-8a2f-fa9eaa8ff210 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.542080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298f5b-dda5-472d-8847-1d3af9fc7a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:52.542732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298f5b-dda5-472d-8847-1d3af9fc7a7a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.543016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298f5b-dda5-472d-8847-1d3af9fc7a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.543294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298f5b-dda5-472d-8847-1d3af9fc7a7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.555606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298f5b-dda5-472d-8847-1d3af9fc7a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-68298f5b-dda5-472d-8847-1d3af9fc7a7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:52.561686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:52.562551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:52.563329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 89/183] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:52.565469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.565469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.567628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.579293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:52.579990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.580833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.581169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.586921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:52.588241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:52.670877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:52.672845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:52.680886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 90/184] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:52.682011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.745973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:52.780713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:52.790676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 95/185] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:52.796233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.804417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f0f0-4eba-4675-8346-1ee5cafc470b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.841834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:52.842589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.842872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.843143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.843802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:52.844102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:52.845514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fe1d3-d483-4895-9bdb-10c3ce992ac9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:52.853805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 91/186] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:52.859722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c91d4-ef24-4faf-bfcd-fa78e9eb5932 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.860012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c91d4-ef24-4faf-bfcd-fa78e9eb5932 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.860313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c91d4-ef24-4faf-bfcd-fa78e9eb5932 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.861555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 92/187] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:52.862256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c91d4-ef24-4faf-bfcd-fa78e9eb5932 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.905514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3eca0-a203-4ba6-a8af-6dfb4aae9683 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.939342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:52.944010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:52.944330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:52.944613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:52.955903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:52.960800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:52.961715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 96/188] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:52.962679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:52.972812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c91d4-ef24-4faf-bfcd-fa78e9eb5932 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:52.987647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.007305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c626ca-976c-4635-b3e5-61f544cefa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.007876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c626ca-976c-4635-b3e5-61f544cefa64 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.008159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c626ca-976c-4635-b3e5-61f544cefa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.008467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c626ca-976c-4635-b3e5-61f544cefa64 None admin] PATH_INFO: `/v3/projects/f2692caece4344a6b60f0019f18d1df0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.009588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c626ca-976c-4635-b3e5-61f544cefa64 None admin] RBAC: Authorizing `identity:get_project(project_id=f2692caece4344a6b60f0019f18d1df0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.011272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c626ca-976c-4635-b3e5-61f544cefa64 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.013122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 93/189] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/projects/f2692caece4344a6b60f0019f18d1df0 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.014809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.015093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.015374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.032937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:53.034143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:53.036872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.068103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b989b-ee37-4b16-91f3-8a8116807297 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.102219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 94/190] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:53.103082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.105381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.106019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.106298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.106574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] PATH_INFO: `/v3/users/6699ac7cb6234a62984c23cde4051e48` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.124720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] RBAC: Authorizing `identity:get_user(user_id=6699ac7cb6234a62984c23cde4051e48)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.126629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.128293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 97/191] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:52 2022] GET /identity/v3/users/6699ac7cb6234a62984c23cde4051e48 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.129054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.213081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba4a06-1165-46a0-823f-fa1ad4a26350 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.213789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.235160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f1149-5bd4-4281-a714-280e29fab3cc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.256124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.257133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.257463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.257681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.258781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.260352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.262225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 95/192] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.262903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.274057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.276924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.277906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.278194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.278622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.279895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:53.280304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.281873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.290086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 98/193] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.290886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.371066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554a175-6f85-43fe-a6c4-1406c7220949 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.398691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bccd3c-197f-407e-9cb3-b84ad59e311a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.407438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.408119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.408437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.408715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.409438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=f2692caece4344a6b60f0019f18d1df0, user.id=2921a61a62e3462084377c936b42cb02) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:53.409746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.411183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.416824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 96/194] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/role_assignments?user.id=2921a61a62e3462084377c936b42cb02&scope.project.id=f2692caece4344a6b60f0019f18d1df0&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.417580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.444695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.445322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.445606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.445880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.459185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.501931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.523010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.524573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.542324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd018ed-7475-4a25-a9db-178d5a5a95ca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.583871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.584538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.584824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.585096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.585771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.587266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.609823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 97/195] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:44:53 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:53.610562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.653053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 99/196] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:53.653951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.710081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.760696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea4a2a-b502-48fe-b680-9e9eaa9bdb5b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.781773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f8687-846f-423b-9505-9cd9e4c69127 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.794708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.805991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.806639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.806917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.807187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] PATH_INFO: `/v3/users/6699ac7cb6234a62984c23cde4051e48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.808360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] RBAC: Authorizing `identity:get_user(user_id=6699ac7cb6234a62984c23cde4051e48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.809935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.813250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 98/197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/users/6699ac7cb6234a62984c23cde4051e48 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.813949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.828157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.830072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.830072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.830072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.839719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:53.844960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:53.845837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:53.846626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 100/198] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:53.847272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:53.942442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e653950-f5ed-4574-bea8-1475c7560e5a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.973169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3850-d994-41c7-ad46-6f90d09632eb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:53.974343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:53.990144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:53.990801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:53.991085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:53.991364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:53.992545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:53.994145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:53.996782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 99/199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:53.997823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.013307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.014517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.015211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.015211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.083327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:54.101448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.103147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.136006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e47a2d-0799-45ae-aaaa-1e8bef7d65c4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.183363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.183984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.184293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.184571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.185266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:54.186354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.187920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.200316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 100/200] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:54.201444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.217385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 101/201] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:54.223077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.226202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:54.335822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f1c50-2af6-4bf4-8b7b-8a7748dcb34c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.339896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b5449-9122-4371-a5c1-1b99f259a8b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.374835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:54.376574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.377322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.377669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.378179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] PATH_INFO: `/v3/users/bc4322dd0c57429bad559e37d5e75137` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.380170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] RBAC: Authorizing `identity:get_user(user_id=bc4322dd0c57429bad559e37d5e75137)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.381727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.383398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 102/202] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/users/bc4322dd0c57429bad559e37d5e75137 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:54.384207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.384404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.384953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.385240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.385515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.386506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=6699ac7cb6234a62984c23cde4051e48) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:54.386815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.389906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.396750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 101/203] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/role_assignments?user.id=6699ac7cb6234a62984c23cde4051e48&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:54.397557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.479053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:54.517086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b37c0-af8e-468d-affd-4184688ef194 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.517612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf569f6a-55fd-4000-b621-5bbe20033c5e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.555287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.555620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.555919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.556173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.556381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.556542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.556688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.556840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.558005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.559487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.561705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 103/204] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:54.567641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.624149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:54.636676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.638156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.678177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5169b53-2153-493b-b588-2bc2056d36cc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.718789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.719417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.719697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.719969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.720757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] RBAC: Adding query filter params (role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=6699ac7cb6234a62984c23cde4051e48) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:54.721121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.722534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.728508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 104/205] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/role_assignments?user.id=6699ac7cb6234a62984c23cde4051e48&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=0f55ac654fb247249b38e3c701826f02 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:54.759324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 102/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:54.765631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:54.770867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:54.891404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343c47a-4996-4a40-a41b-9cd2638e6c17 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:54.929182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c32a3e-4987-4e62-a6a5-39e581642efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:54.929874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c32a3e-4987-4e62-a6a5-39e581642efb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:54.930158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c32a3e-4987-4e62-a6a5-39e581642efb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:54.930435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c32a3e-4987-4e62-a6a5-39e581642efb None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:54.931769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c32a3e-4987-4e62-a6a5-39e581642efb None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:54.933420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c32a3e-4987-4e62-a6a5-39e581642efb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:54.936132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 105/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:54.944713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:55.065840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aea00-a7de-4cd7-9ecd-483818f9730d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:55.073962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:55.121573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:55.122191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:55.122470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:55.122788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:55.123728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] RBAC: Adding query filter params (role.id=6f21288e54a24472b723fe127cae383e, scope.project.id=b59720c712524d768e1fc85c3cf651b5, user.id=bc4322dd0c57429bad559e37d5e75137) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:55.124039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:55.125511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a6abe-2a1e-4960-8ff2-71e4c6a38070 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:55.136436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 103/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:54 2022] GET /identity/v3/role_assignments?user.id=bc4322dd0c57429bad559e37d5e75137&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=6f21288e54a24472b723fe127cae383e => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:56.167291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a3e4d-f387-4def-9482-4c84c0f12a2d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.168293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a3e4d-f387-4def-9482-4c84c0f12a2d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.168779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a3e4d-f387-4def-9482-4c84c0f12a2d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.170974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 106/209] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:56.177193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c4a3-2457-450e-a43e-9a60f4b4bf16 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.178208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c4a3-2457-450e-a43e-9a60f4b4bf16 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.178713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c4a3-2457-450e-a43e-9a60f4b4bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.199461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4a95c4a3-2457-450e-a43e-9a60f4b4bf16 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:56.201108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a95c4a3-2457-450e-a43e-9a60f4b4bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:56.275122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 104/210] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:56.311961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec52808-856a-464a-b351-254a037a3bb0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.312657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec52808-856a-464a-b351-254a037a3bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.313139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec52808-856a-464a-b351-254a037a3bb0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.314943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 107/211] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:56.320127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.320448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.320722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.339051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:56.340221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:56.408891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 105/212] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:56.558836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec52808-856a-464a-b351-254a037a3bb0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:56.636607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:56.669198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec52808-856a-464a-b351-254a037a3bb0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:56.698124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:56.706331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:56.706975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.707255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.707526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.715496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:56.720908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:56.727826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:56.728747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:44:56.730393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 108/213] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:56.735441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:56.839279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52379ca-174d-42e0-839e-24b2ef0c2cc4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:56.862187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f190a-120e-4342-aa78-da35f60b1106 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:56.881571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:56.882542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.883018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.883510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.884889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:56.886867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:56.893675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 106/214] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:56.895696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:56.896541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:56.905162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:56.905777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:56.906056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:56.906327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:56.906996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:56.907297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:56.909300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:56.920951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 109/215] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:44:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:56.929524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:56.990240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:57.010812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4228a-b70a-40ff-a024-13d0399dbec4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.046506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c8f2a-a635-473a-83e8-f726e7e03122 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.049283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6361f36-f916-454b-94e1-5c4c00cbc836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.049903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f36-f916-454b-94e1-5c4c00cbc836 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.050238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f36-f916-454b-94e1-5c4c00cbc836 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.050460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f36-f916-454b-94e1-5c4c00cbc836 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.051115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361f36-f916-454b-94e1-5c4c00cbc836 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:57.052583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361f36-f916-454b-94e1-5c4c00cbc836 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:57.079189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 107/216] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:44:56 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:44:57.079189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281da6-42d0-4c6b-b128-cea69dc18f80 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.079189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281da6-42d0-4c6b-b128-cea69dc18f80 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.079189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281da6-42d0-4c6b-b128-cea69dc18f80 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.079189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 108/217] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:57.086327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe601-12b9-4b60-b124-8f00ab472442 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.086639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe601-12b9-4b60-b124-8f00ab472442 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.086882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fe601-12b9-4b60-b124-8f00ab472442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.094157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.095004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.095406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.095756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] PATH_INFO: `/v3/users/894c7f373c4847698e4d2ab87a642529` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.112952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-da9fe601-12b9-4b60-b124-8f00ab472442 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:57.114128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] RBAC: Authorizing `identity:get_user(user_id=894c7f373c4847698e4d2ab87a642529)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:57.114622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-da9fe601-12b9-4b60-b124-8f00ab472442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:57.115671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:57.118072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 110/218] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:56 2022] GET /identity/v3/users/894c7f373c4847698e4d2ab87a642529 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:57.124603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:57.189605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:57.200026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 109/219] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:44:57.212883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b110-03be-4f6d-b4b6-980f244adb79 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.213368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b110-03be-4f6d-b4b6-980f244adb79 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.213779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1870b110-03be-4f6d-b4b6-980f244adb79 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.215361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 110/220] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:44:57.222474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.222897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.223316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.244901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:57.246426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:57.254221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb4b4d-9f2d-417d-bdde-67fbcf9b1a3c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.298088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.298706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.299053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.299261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.322902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:57.330046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 111/221] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:44:57.386888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:57.389059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:57.440690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:57.489985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:57.508818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 111/222] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:44:57.515814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:57.623689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470e1cb-620e-4595-a382-9edc896ffc56 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.653583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd92d70-27c8-4d76-ae22-9bb610b73728 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.670425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.671073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.671357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.671674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.686933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:57.691928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:57.692869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:44:57.693687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:57.694288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.694288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.694426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 112/223] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:44:57.694490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.694490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.694611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:57.695592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:57.697178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:57.699543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 112/224] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:44:57.709716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:57.736596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:57.762469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:57.813164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2bb49-c4d6-414b-9a75-11975eead311 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.849218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d967bf-f788-4077-a4f4-dbac959d1062 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:57.852699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.852988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.853340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.853670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.866089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:44:57.871668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:44:57.873330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:44:57.874496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:44:57.875477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 113/225] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:44:57.876456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:57.897193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:57.898475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:57.898960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:57.899404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:57.900340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=894c7f373c4847698e4d2ab87a642529) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:57.900824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:57.903214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:57.920562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 113/226] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/role_assignments?user.id=894c7f373c4847698e4d2ab87a642529&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:57.920562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.062985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f921-60a5-477a-b3af-de64fedb36f0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.094534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:58.103483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:58.125707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998a10-40cc-4b58-8229-c4fa985b1cf1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.137283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.138278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.138278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.139012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.139644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:58.139950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.141865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.152742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 114/227] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:58.158311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.171076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.172104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.172618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.173089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.173951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:44:58.174523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.176905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.186701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 114/228] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:44:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:58.192569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.315152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b4d9f-4f7a-403b-87f1-f25c902d1ecd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.324522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba749c22-b008-492c-90d3-06e5fb1c6866 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.373333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.373977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.374256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.374527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] PATH_INFO: `/v3/users/6699ac7cb6234a62984c23cde4051e48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.375812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] RBAC: Authorizing `identity:get_user(user_id=6699ac7cb6234a62984c23cde4051e48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.377271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.384185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 115/229] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:58 2022] GET /identity/v3/users/6699ac7cb6234a62984c23cde4051e48 => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:58.387171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.390109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.390761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.391037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.391308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] PATH_INFO: `/v3/users/6699ac7cb6234a62984c23cde4051e48` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.392721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] RBAC: Authorizing `identity:get_user(user_id=6699ac7cb6234a62984c23cde4051e48)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.395957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.397569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 115/230] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:44:58 2022] GET /identity/v3/users/6699ac7cb6234a62984c23cde4051e48 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:58.403082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.431448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:58.495267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:58.556035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12a932-cd35-4a4d-9d32-bfbfb94df37a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.583432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ace30-2b3e-464c-9a9a-7ddfb95329c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.625468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.626400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.626816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.627225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.628559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.631692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.637668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 116/231] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:58 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:58.641292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.660869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.661923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.662349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.662762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.665665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.668442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.672355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 116/232] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:44:58 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:44:58.687344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:44:58.811258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb865e4-fe22-4f34-b8a4-b7e5071d952c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.830601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:58.848740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:44:58.860584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57353ccf-5ca3-4711-98a6-f13073d4afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.863848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57353ccf-5ca3-4711-98a6-f13073d4afe7 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.864363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57353ccf-5ca3-4711-98a6-f13073d4afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.864781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57353ccf-5ca3-4711-98a6-f13073d4afe7 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072/users/6699ac7cb6234a62984c23cde4051e48/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.868130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57353ccf-5ca3-4711-98a6-f13073d4afe7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3991445c4c64256be54a26d63fb7072, user_id=6699ac7cb6234a62984c23cde4051e48, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.870914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57353ccf-5ca3-4711-98a6-f13073d4afe7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.891066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491577e7-1c43-443a-9f26-67ef4fcd5926 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:44:58.895031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 117/233] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:58 2022] PUT /identity/v3/projects/e3991445c4c64256be54a26d63fb7072/users/6699ac7cb6234a62984c23cde4051e48/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:44:58.953861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb29cc-1163-42e7-8c71-080cca6687cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:44:58.954822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb29cc-1163-42e7-8c71-080cca6687cb None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:58.955243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb29cc-1163-42e7-8c71-080cca6687cb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:58.955673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb29cc-1163-42e7-8c71-080cca6687cb None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072/users/6699ac7cb6234a62984c23cde4051e48/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:58.959043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb29cc-1163-42e7-8c71-080cca6687cb None admin] RBAC: Authorizing `identity:create_grant(project_id=e3991445c4c64256be54a26d63fb7072, user_id=6699ac7cb6234a62984c23cde4051e48, role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:44:58.960728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb29cc-1163-42e7-8c71-080cca6687cb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:44:58.977367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 117/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:44:58 2022] PUT /identity/v3/projects/e3991445c4c64256be54a26d63fb7072/users/6699ac7cb6234a62984c23cde4051e48/roles/0f55ac654fb247249b38e3c701826f02 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:44:59.743598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd47ef-9c37-42ca-b249-53b822319c44 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:59.744629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd47ef-9c37-42ca-b249-53b822319c44 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:59.745113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd47ef-9c37-42ca-b249-53b822319c44 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:59.746649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 118/235] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:44:59.753428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e09-807d-4ea1-8c03-aadb383ae667 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:44:59.754252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e09-807d-4ea1-8c03-aadb383ae667 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:44:59.754734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e09-807d-4ea1-8c03-aadb383ae667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:44:59.791726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-38521e09-807d-4ea1-8c03-aadb383ae667 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:44:59.792937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-38521e09-807d-4ea1-8c03-aadb383ae667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:44:59.874461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 118/236] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:00.118230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508444a2-f887-4206-a04d-68831bd66115 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.118823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508444a2-f887-4206-a04d-68831bd66115 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.119329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508444a2-f887-4206-a04d-68831bd66115 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.120889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 119/237] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:00.126340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.126875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.127359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.156944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:00.158427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:00.204804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508444a2-f887-4206-a04d-68831bd66115 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:00.276705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:00.289643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 119/238] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:00.344875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508444a2-f887-4206-a04d-68831bd66115 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:00.382118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:00.383106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.383597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.384063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.395997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:00.401797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:00.402653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:00.403417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:00.406242 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 120/239] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:00.412179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:00.546420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d8c1c-aac5-4042-97af-eeeb9ae530a8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:00.595750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:00.596725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.597239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.597702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.598641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:00.599143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:00.600741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f074-2fec-4c4d-b849-e0d90bce5c80 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:00.615731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 120/240] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:00.621539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:00.636455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239850a-d255-4cb9-bfc1-c16d350f2911 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.636455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239850a-d255-4cb9-bfc1-c16d350f2911 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.636455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239850a-d255-4cb9-bfc1-c16d350f2911 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.636794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 121/241] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:00.637462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239850a-d255-4cb9-bfc1-c16d350f2911 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:00.666761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:00.719664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:00.752030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8e914-53ab-41c3-bd37-e3190e968bbf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:00.771640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239850a-d255-4cb9-bfc1-c16d350f2911 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:00.792528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675235d-0c13-4084-9af8-0d001cd0b298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:00.793483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675235d-0c13-4084-9af8-0d001cd0b298 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.793930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675235d-0c13-4084-9af8-0d001cd0b298 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.794348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675235d-0c13-4084-9af8-0d001cd0b298 None admin] PATH_INFO: `/v3/users/b52a57c7ae0f4ab5bc19fc97d24542d4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.811686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675235d-0c13-4084-9af8-0d001cd0b298 None admin] RBAC: Authorizing `identity:get_user(user_id=b52a57c7ae0f4ab5bc19fc97d24542d4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:00.813795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675235d-0c13-4084-9af8-0d001cd0b298 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:00.814002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:00.814649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.814981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.815264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.816407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:00.816676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 121/242] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:00 2022] GET /identity/v3/users/b52a57c7ae0f4ab5bc19fc97d24542d4 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:00.818052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.818477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.818945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:00.822415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:00.824195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 122/243] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:00.828962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:00.845157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:00.846361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:00.930145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 122/244] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:00.930910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:00.943351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283bed1-4180-4438-9228-e2d870db953c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:00.958348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:00.978777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:00.986366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:00.987351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:00.987880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:00.988372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.089041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f3e19-8bcf-46a6-8179-5202da84acd5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:01.116771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:01.119775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:01.177534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:01.178163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.178443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.178712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.179366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:01.180951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:01.228988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 123/245] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:00 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:01.240341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 123/246] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:01.245889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:01.315288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:01.382618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:01.388100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:01.428695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2600f4-fe46-42bb-847e-f5667c2a7a3f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:01.486637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:01.487283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.487562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.487836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.489298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:01.490918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:01.492646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 124/247] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:01 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:01.498847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:01.506437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfab202-f554-4d48-8ae5-7eb9ad90d3d9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:01.554747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b340f4-d6e6-4ec7-83df-489bee183ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:01.555542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b340f4-d6e6-4ec7-83df-489bee183ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.555820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b340f4-d6e6-4ec7-83df-489bee183ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.556500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b340f4-d6e6-4ec7-83df-489bee183ad9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.571359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b340f4-d6e6-4ec7-83df-489bee183ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-94b340f4-d6e6-4ec7-83df-489bee183ad9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:01.579168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:01.580077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:01.580960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 124/248] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:01.582085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64aaa-53cf-4868-a711-e924576f756f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.582085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64aaa-53cf-4868-a711-e924576f756f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.583708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a64aaa-53cf-4868-a711-e924576f756f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.583708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 125/249] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:01.589977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.590256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.590527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.613505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:01.614732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:01.628402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f34459-487a-4fdc-92c1-57206540b7b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:01.669119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:01.670049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.670330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.670658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.671394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] RBAC: Adding query filter params (role.id=6f21288e54a24472b723fe127cae383e, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=b52a57c7ae0f4ab5bc19fc97d24542d4) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:01.671705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:01.673309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:01.679317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 125/250] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:01 2022] GET /identity/v3/role_assignments?user.id=b52a57c7ae0f4ab5bc19fc97d24542d4&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=6f21288e54a24472b723fe127cae383e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:01.680068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:01.707298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:01.723211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 126/251] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:01.815921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deca75c-db4a-471d-bc13-c54470afa506 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:01.859713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:01.860471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:01.860759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:01.861038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:01.861722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:01.862032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:01.863466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737566f1-7c73-4617-b6b2-60fe7d1bcb01 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:01.873967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 126/252] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:01.884474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:01.932721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:02.012572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f244c0f-e033-46cd-ab3a-0cc4c3c79463 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.012886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f244c0f-e033-46cd-ab3a-0cc4c3c79463 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.013180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f244c0f-e033-46cd-ab3a-0cc4c3c79463 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.014718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 127/253] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:02.020312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.020845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.021493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.042845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:02.046189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:02.048400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e2b2d-df5c-47f8-9319-191702a48864 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:02.089679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:02.090700 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.091176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.092437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] PATH_INFO: `/v3/users/894c7f373c4847698e4d2ab87a642529` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.096156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] RBAC: Authorizing `identity:get_user(user_id=894c7f373c4847698e4d2ab87a642529)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:02.101127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:02.101127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 127/254] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:01 2022] GET /identity/v3/users/894c7f373c4847698e4d2ab87a642529 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:02.101357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:02.207093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:02.222638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369dfcb7-d173-4083-b354-c0b49f653ade None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:02.254521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 128/255] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:02.255413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:02.268617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:02.273693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:02.274754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.275044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.275336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.295839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:02.311791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:02.312755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:02.313538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 128/256] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:02.328040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:02.496602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fabcb-4108-47d4-a61d-85c6053a1733 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:02.507020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e51b-10f5-42c3-b897-6468933b62c8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:02.567821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:02.568513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.568795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.569068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.570410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:02.571185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:02.571861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:02.572300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.572794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.573262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.573842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 129/257] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:02 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:02.574325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:02.574821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:02.576736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:02.584296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:02.593408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 129/258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:02.599371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:02.624648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:02.707139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:02.763019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ce54-10cc-44e4-bf25-f84457a71b2a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:02.804473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c9b8e-afd8-4a02-b612-4e1cad0549cb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:02.823686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:02.824383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.824663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.824932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072/users/894c7f373c4847698e4d2ab87a642529/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.831518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] RBAC: Authorizing `identity:create_grant(project_id=e3991445c4c64256be54a26d63fb7072, user_id=894c7f373c4847698e4d2ab87a642529, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:02.832991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:02.861675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 130/259] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:02 2022] PUT /identity/v3/projects/e3991445c4c64256be54a26d63fb7072/users/894c7f373c4847698e4d2ab87a642529/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:02.862562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:02.881964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:02.882676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:02.882960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:02.883231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] PATH_INFO: `/v3/users/6699ac7cb6234a62984c23cde4051e48` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:02.885027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] RBAC: Authorizing `identity:get_user(user_id=6699ac7cb6234a62984c23cde4051e48)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:02.887639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:02.892770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 130/260] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:02 2022] GET /identity/v3/users/6699ac7cb6234a62984c23cde4051e48 => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:02.897900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:03.062969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:03.091831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd0776-e454-4793-8d6a-7d85668dfa7e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:03.130380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817e090-fdbe-4a0e-a54e-f4b31855117d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:03.177844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:03.190113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:03.190768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:03.191052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:03.191327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:03.209238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:03.214202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:03.215079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:03.215858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 131/261] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:03.223945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:03.224615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:03.224900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:03.225233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:03.226515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:03.347851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:03.353488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:03.458973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb903f-1807-44c3-a5fa-52f6baca5d40 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:03.483620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:03.524051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 131/262] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:03.527865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:03.528517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:03.529401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:03.529831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:03.531529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:03.539568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:03.539947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:03.541414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:03.557081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 132/263] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:03.562931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:03.657398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:03.728080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d487d-8998-4951-b4b1-56cca183486e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:03.735723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4928cdc-1e16-4466-b4b0-064a2705aa36 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:03.796687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:03.797340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:03.797624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:03.797900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] PATH_INFO: `/v3/users/6699ac7cb6234a62984c23cde4051e48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:03.804960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] RBAC: Authorizing `identity:get_user(user_id=6699ac7cb6234a62984c23cde4051e48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:03.806419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:03.808080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 133/264] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:03 2022] GET /identity/v3/users/6699ac7cb6234a62984c23cde4051e48 => generated 283 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:03.811126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:03.811774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:03.812056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:03.812349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:03.814040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:03.814538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:03.815499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:03.822793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 132/265] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:03 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:03.823907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:03.930801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:04.015434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4c4fb-68fd-4f08-bcd6-9156e547f8a1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:04.052406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:04.066521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2acc4-6dac-4811-8171-20a20b2800b0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:04.076939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:04.077570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.077862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.078135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.202238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:04.203541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:04.224994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:04.225626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.225908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.226182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.226927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] RBAC: Adding query filter params (role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=6699ac7cb6234a62984c23cde4051e48) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:04.227237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:04.228724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:04.258649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 133/266] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:03 2022] GET /identity/v3/role_assignments?user.id=6699ac7cb6234a62984c23cde4051e48&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=0f55ac654fb247249b38e3c701826f02 => generated 608 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:04.348370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 134/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:04.358086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:04.508593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b2306-cf5c-40e6-99f9-6391a932d6b0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:04.526689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:04.556931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2438e-59d5-4a48-9528-7b71b6903888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:04.557574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2438e-59d5-4a48-9528-7b71b6903888 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.557857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2438e-59d5-4a48-9528-7b71b6903888 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.558133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2438e-59d5-4a48-9528-7b71b6903888 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.565459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2438e-59d5-4a48-9528-7b71b6903888 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:04.566972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2438e-59d5-4a48-9528-7b71b6903888 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:04.568940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 134/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:04 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:04.575299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:04.595254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad024b5c-687e-4a52-8e5c-5434a9be1582 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.595254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad024b5c-687e-4a52-8e5c-5434a9be1582 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.595542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad024b5c-687e-4a52-8e5c-5434a9be1582 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.597636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 135/269] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:04.603325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0ee7f-4c76-4481-8b51-9f773c6d408b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.603637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0ee7f-4c76-4481-8b51-9f773c6d408b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.603926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0ee7f-4c76-4481-8b51-9f773c6d408b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.621384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:04.649466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c9a0ee7f-4c76-4481-8b51-9f773c6d408b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:04.650722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c9a0ee7f-4c76-4481-8b51-9f773c6d408b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:04.745234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454424a-8f9c-4e8b-bb04-b26280d7e09c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:04.755990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 136/270] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:04.784483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef4a7-5686-4c3f-94c3-2df775550196 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.784767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef4a7-5686-4c3f-94c3-2df775550196 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.784976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceef4a7-5686-4c3f-94c3-2df775550196 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.786828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 137/271] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:04.792210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.792681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.793021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.805005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:04.805632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.805914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.806192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.806921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=6699ac7cb6234a62984c23cde4051e48) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:04.807234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:04.808719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124456d8-15dc-4d89-9365-069364c2efdb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:04.814554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:04.815776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:04.815961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 135/272] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:04 2022] GET /identity/v3/role_assignments?user.id=6699ac7cb6234a62984c23cde4051e48&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:04.817178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8c014-7830-4e4b-9d23-05e3ca0e0ca9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.817468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8c014-7830-4e4b-9d23-05e3ca0e0ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.817742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8c014-7830-4e4b-9d23-05e3ca0e0ca9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.819077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 136/273] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:04.830429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:04.830878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:04.831259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:04.868950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:04.870636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:04.958728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 138/274] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:05.002349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 137/275] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:05.091034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.146036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.214560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47b6e0-757c-43ae-818f-2e2cd2801e25 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.250615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.251580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.252062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.252572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.264265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:05.269458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:05.270332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:05.271241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:05.273854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 139/276] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:05.279920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.302075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.314001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.408077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486aed2-85a4-464b-ab39-8b7499568bc3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.432468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.437195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24525e-addf-4205-befd-1deb798235c2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.446623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.447244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.447526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.448008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.448501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:05.448780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:05.450157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:05.459707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 138/277] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:05.459707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.480978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.481614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.482084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.482448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.483694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:05.485204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:05.486981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 140/278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:05.488136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.550587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.575162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e393d-86ff-42e0-bbf1-5734727d2f7c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.604770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c13c13-d1dd-4167-8826-254eab4f4238 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.610865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.611502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.611787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.612063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.622311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:05.628215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:05.629108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:05.629874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:05.634045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 139/279] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:05.634844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.648444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.649875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.650369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.650839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] PATH_INFO: `/v3/users/6a64a7c58cd4496f82e50ea846090eca` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.664959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=6a64a7c58cd4496f82e50ea846090eca)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:05.666305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:05.668291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 141/280] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/users/6a64a7c58cd4496f82e50ea846090eca => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:05.669059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.696719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.755296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2c61b-b12b-4cad-b893-45f14a41271b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.783813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4c7bd-9547-406e-ad3f-3f440bbda1e6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.789761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.790372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.790649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.790925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.791569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:05.793036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:05.803601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.817862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 140/281] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:05 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:05.818492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.843305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.844039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.844338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.844613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.845302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:05.845624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:05.847050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8cb11e-69e3-4b08-89e8-10b5cd90cf56 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:05.860569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 142/282] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:05.860569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7378-019d-48cc-b1f8-77a2dcc4dabf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.860569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7378-019d-48cc-b1f8-77a2dcc4dabf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.860569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7378-019d-48cc-b1f8-77a2dcc4dabf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.861031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 143/283] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:05.862661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7378-019d-48cc-b1f8-77a2dcc4dabf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:05.939679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fca325-b422-4882-b6f7-7e65bcebd9f8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.956918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:05.974444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7378-019d-48cc-b1f8-77a2dcc4dabf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:05.978502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:05.979137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.979425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.979698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:05.980473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, user.id=6a64a7c58cd4496f82e50ea846090eca, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:05.980795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:05.982878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06999ef-599a-4ca3-845a-90c449862a51 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:05.994864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 141/284] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/role_assignments?user.id=6a64a7c58cd4496f82e50ea846090eca&scope.system=all&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:05.994864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:05.994864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:05.994864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.014970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:06.016202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:06.025656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:06.026314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:06.026601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:06.026877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] PATH_INFO: `/v3/users/b52a57c7ae0f4ab5bc19fc97d24542d4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.028904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=b52a57c7ae0f4ab5bc19fc97d24542d4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:06.030407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:06.032310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 144/285] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:05 2022] GET /identity/v3/users/b52a57c7ae0f4ab5bc19fc97d24542d4 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:06.039162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:06.094998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:06.118800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 142/286] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:06.173492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e1dadc-1f59-41ae-9d60-e992a896b3b3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:06.211182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:06.211826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:06.212139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:06.212401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.213542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:06.214987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:06.217585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 145/287] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:06 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:06.223551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:06.340073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d62ff-c115-4422-b3e5-84505d128aec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:06.387065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:06.387753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:06.388086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:06.388899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072/users/b52a57c7ae0f4ab5bc19fc97d24542d4/roles/6f21288e54a24472b723fe127cae383e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.392204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] RBAC: Authorizing `identity:create_grant(project_id=e3991445c4c64256be54a26d63fb7072, user_id=b52a57c7ae0f4ab5bc19fc97d24542d4, role_id=6f21288e54a24472b723fe127cae383e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:06.393688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:06.401544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:06.411921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 143/288] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:06 2022] PUT /identity/v3/projects/e3991445c4c64256be54a26d63fb7072/users/b52a57c7ae0f4ab5bc19fc97d24542d4/roles/6f21288e54a24472b723fe127cae383e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:06.432845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:06.530840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39565f-ff6d-47c7-961b-b0bb4d9b87df None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:06.579550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:06.580212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:06.580518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:06.580791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.596616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:06.602334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:06.603245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:06.604081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 146/289] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:06.610139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:06.665373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:06.721843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e316a20-3aab-4fff-8332-f65d9bb0115c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:06.756862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:06.757550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:06.757840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:06.758111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.758771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:06.759074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:06.760526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:06.770110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 144/290] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:06.775609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:06.880091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:06.902700 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74c030-9876-4cc6-b584-ba7c2e6b1231 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:06.941175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:06.941908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:06.942193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:06.942470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] PATH_INFO: `/v3/users/894c7f373c4847698e4d2ab87a642529` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:06.943607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] RBAC: Authorizing `identity:get_user(user_id=894c7f373c4847698e4d2ab87a642529)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:06.946805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:06.948412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 147/291] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:06 2022] GET /identity/v3/users/894c7f373c4847698e4d2ab87a642529 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:06.954194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:07.074901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ec37-05b4-494c-985b-dfb3f4960d14 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:07.088958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:07.115098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:07.115763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:07.116051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:07.116351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:07.188815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:07.190216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:07.295616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 145/292] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:45:07.301345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:07.426744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774da1d-04f7-4922-939c-9721f5957b4b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:07.469269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb6bb6-ed2c-4f9f-9abb-f0f33617cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:07.469916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6bb6-ed2c-4f9f-9abb-f0f33617cd77 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:07.470198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6bb6-ed2c-4f9f-9abb-f0f33617cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:07.470486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb6bb6-ed2c-4f9f-9abb-f0f33617cd77 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:07.471612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb6bb6-ed2c-4f9f-9abb-f0f33617cd77 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:07.473078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb6bb6-ed2c-4f9f-9abb-f0f33617cd77 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:07.474712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 148/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:07 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:07.480800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:07.495333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:07.592675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36580fbe-36dd-4996-ace1-8a1e225bf7b7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:07.627579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:07.628202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:07.628518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:07.628795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:07.629527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=894c7f373c4847698e4d2ab87a642529) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:07.629835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:07.631323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5935f0-9a62-4529-bd51-9cc7374f5efe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:07.637923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 146/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:07 2022] GET /identity/v3/role_assignments?user.id=894c7f373c4847698e4d2ab87a642529&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:08.639365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520762bb-693f-4a4c-85bf-52c4af941c2a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:08.639878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520762bb-693f-4a4c-85bf-52c4af941c2a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:08.639878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520762bb-693f-4a4c-85bf-52c4af941c2a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:08.641602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 149/295] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:08.650222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a5b0-cd99-4bdc-a7c1-a64c72ac7824 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:08.650988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef2f83-8bdd-4225-8069-bc020a000f4e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:08.652860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef2f83-8bdd-4225-8069-bc020a000f4e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:08.652987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a5b0-cd99-4bdc-a7c1-a64c72ac7824 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:08.653097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef2f83-8bdd-4225-8069-bc020a000f4e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:08.653194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a5b0-cd99-4bdc-a7c1-a64c72ac7824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:08.653490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 150/296] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:08.658800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c07314-9fe3-4cfb-90f4-7612585f166f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:08.659278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c07314-9fe3-4cfb-90f4-7612585f166f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:08.659738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c07314-9fe3-4cfb-90f4-7612585f166f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:08.671731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0928a5b0-cd99-4bdc-a7c1-a64c72ac7824 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:08.673066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0928a5b0-cd99-4bdc-a7c1-a64c72ac7824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:08.678603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-34c07314-9fe3-4cfb-90f4-7612585f166f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:08.680108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-34c07314-9fe3-4cfb-90f4-7612585f166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:08.712657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:08.748714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 147/297] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:08.760541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 151/298] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:08.841326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422269-c46c-4f3a-9b5e-e9a023b32897 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:08.841907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422269-c46c-4f3a-9b5e-e9a023b32897 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:08.842327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422269-c46c-4f3a-9b5e-e9a023b32897 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:08.844089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 148/299] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:08.849331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:08.849606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:08.849882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:08.868722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:08.869913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:08.939948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 152/300] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:09.024856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52422269-c46c-4f3a-9b5e-e9a023b32897 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.030064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.051278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.135972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52422269-c46c-4f3a-9b5e-e9a023b32897 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.139969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.156522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699cfb-0d9d-42a7-ac5b-b8527a055454 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.175222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba81ce3-4551-4f5a-9305-29335e79a0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.175862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba81ce3-4551-4f5a-9305-29335e79a0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.176145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba81ce3-4551-4f5a-9305-29335e79a0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.176449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba81ce3-4551-4f5a-9305-29335e79a0a4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.189665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba81ce3-4551-4f5a-9305-29335e79a0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:09.192169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba75d59-c177-4019-8e82-0922ace2ae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.192776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba75d59-c177-4019-8e82-0922ace2ae98 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.193057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba75d59-c177-4019-8e82-0922ace2ae98 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.193329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba75d59-c177-4019-8e82-0922ace2ae98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6ba81ce3-4551-4f5a-9305-29335e79a0a4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:09.194513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:09.195354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 149/301] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:09.197104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dcb33-5368-4286-9298-bdc57eefad40 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.197800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dcb33-5368-4286-9298-bdc57eefad40 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.197800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dcb33-5368-4286-9298-bdc57eefad40 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.198999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 150/302] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:09.201577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dcb33-5368-4286-9298-bdc57eefad40 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.214310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba75d59-c177-4019-8e82-0922ace2ae98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2ba75d59-c177-4019-8e82-0922ace2ae98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:09.219732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:09.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:09.221468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 153/303] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:09.222335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.222610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.222878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.240678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:09.241894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:09.270611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.307930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 154/304] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:09.308801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.317172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dcb33-5368-4286-9298-bdc57eefad40 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.353127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.354133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.354417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.354690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.355338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:09.355997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.358562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.367422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 151/305] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:09.368144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.385986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.424029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4037a25-1db6-4f95-9c23-d1ea3768c3a1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.459160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.459795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.460079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.460381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.461822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.463014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.468334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 155/306] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:09.468334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.479153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dc635-3e30-4fb3-a9c7-157b9fa35179 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.507519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.517789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.518534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.518898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.519247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.519973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:09.520382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.521820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.531299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 152/307] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:09.532082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.571804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02271e-5658-4074-bfaf-597f64fea254 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.605126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.605750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.606027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.606295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] PATH_INFO: `/v3/users/ca50d24d44ea4654994e5c59a6f503e5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.611698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.622736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ca50d24d44ea4654994e5c59a6f503e5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.624195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.626046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 156/308] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/users/ca50d24d44ea4654994e5c59a6f503e5 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:09.626624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.639751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84a4cd-662d-45aa-ba22-05f1b798233b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.675687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.676355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.676639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.676885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] PATH_INFO: `/v3/groups` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.677512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.678942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.695296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 153/309] 10.222.0.57 () {60 vars in 1134 bytes} [Thu Sep 15 12:45:09 2022] POST /identity/v3/groups => generated 229 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:09.696540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.737734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6ca96-b0f7-4859-9330-6730f70059d4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.762588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.777537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.778316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.778528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.779812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] PATH_INFO: `/v3/users/6a64a7c58cd4496f82e50ea846090eca` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.781190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] RBAC: Authorizing `identity:get_user(user_id=6a64a7c58cd4496f82e50ea846090eca)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.782664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.784528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 157/310] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/users/6a64a7c58cd4496f82e50ea846090eca => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:09.785323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.824116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c29e0-dca4-4955-bbfa-1d83c0b1d094 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.855274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:09.862950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.863612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.863898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.864179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.875597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:09.881036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:09.881893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:09.882670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 154/311] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:09.883343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:09.912025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3046a5c-bd94-4a67-8594-40e2d319184e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:09.947333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:09.947944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:09.948225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:09.948526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:09.949258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] RBAC: Adding query filter params (role.id=6f21288e54a24472b723fe127cae383e, user.id=ca50d24d44ea4654994e5c59a6f503e5, scope.system=all) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:09.949567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:09.951383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:09.956541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 158/312] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/role_assignments?user.id=ca50d24d44ea4654994e5c59a6f503e5&scope.system=all&role.id=6f21288e54a24472b723fe127cae383e => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:09.957591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:10.014748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245eb35-0845-4ef4-b5d4-37231c4220ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:10.026065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:10.059414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:10.060621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:10.060999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:10.061349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] PATH_INFO: `/v3/system/users/6a64a7c58cd4496f82e50ea846090eca/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:10.063750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a64a7c58cd4496f82e50ea846090eca, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:10.065691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:10.078385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 155/313] 10.222.0.57 () {58 vars in 1334 bytes} [Thu Sep 15 12:45:09 2022] PUT /identity/v3/system/users/6a64a7c58cd4496f82e50ea846090eca/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:10.079081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1206bf5-638f-48e5-bfdd-2d38d87d45dc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:10.125417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:10.136720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:10.137371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:10.137653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:10.138047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:10.139051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:10.139356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:10.141121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:10.151788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 159/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:10.158482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:10.275361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff7e6e-2e26-4d9b-aa8e-91d024d606b4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:10.310291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:10.311003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:10.311385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:10.311557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] PATH_INFO: `/v3/users/b52a57c7ae0f4ab5bc19fc97d24542d4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:10.312793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] RBAC: Authorizing `identity:get_user(user_id=b52a57c7ae0f4ab5bc19fc97d24542d4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:10.314252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:10.316298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 156/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:10 2022] GET /identity/v3/users/b52a57c7ae0f4ab5bc19fc97d24542d4 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:10.321931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:10.429035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359aa7a7-3a8f-47e4-b047-558f3921d574 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:10.465596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:10.466836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:10.467210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:10.467494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:10.529451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:10.543499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:10.546165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:10.652661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 160/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:10.658117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:10.740620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:10.770796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c3118-3688-4ff6-bf28-ba34e51b5839 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:10.805661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eca6e-f973-4c4d-abd4-c966259ccff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:10.806220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eca6e-f973-4c4d-abd4-c966259ccff3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:10.806502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eca6e-f973-4c4d-abd4-c966259ccff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:10.806778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eca6e-f973-4c4d-abd4-c966259ccff3 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:10.807893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eca6e-f973-4c4d-abd4-c966259ccff3 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:10.809481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eca6e-f973-4c4d-abd4-c966259ccff3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:10.812163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 157/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:10 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:10.818098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:10.912211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:10.926100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e35ea-7963-4642-a6ca-cddb248666f1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:10.964113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:10.965074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:10.965560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:10.966541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:10.967499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] RBAC: Adding query filter params (role.id=6f21288e54a24472b723fe127cae383e, scope.project.id=e3991445c4c64256be54a26d63fb7072, user.id=b52a57c7ae0f4ab5bc19fc97d24542d4) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:10.968013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:10.969729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd2819-351c-40c3-a4ec-6eeaadeeb63d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:10.975877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 161/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:10 2022] GET /identity/v3/role_assignments?user.id=b52a57c7ae0f4ab5bc19fc97d24542d4&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=6f21288e54a24472b723fe127cae383e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:12.019725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be11b7f-75d0-4018-99d3-300f692a3247 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.020243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be11b7f-75d0-4018-99d3-300f692a3247 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.020628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be11b7f-75d0-4018-99d3-300f692a3247 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.021672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 158/319] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:12.026856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025757e-e0b5-47a4-975f-770bacbc5ca0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.027256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025757e-e0b5-47a4-975f-770bacbc5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.027437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025757e-e0b5-47a4-975f-770bacbc5ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.046543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3025757e-e0b5-47a4-975f-770bacbc5ca0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:12.047741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3025757e-e0b5-47a4-975f-770bacbc5ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:12.123487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 162/320] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:12.248571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bd09d-6cb3-4782-bad0-baa9ecf1a0e7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.248908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bd09d-6cb3-4782-bad0-baa9ecf1a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.249128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bd09d-6cb3-4782-bad0-baa9ecf1a0e7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.250590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 159/321] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:12.255636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.255924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.256192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.275723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:12.276836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:12.309352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3455c87-6bca-43e2-a90c-5bfa14c7f245 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.310168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3455c87-6bca-43e2-a90c-5bfa14c7f245 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.310443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3455c87-6bca-43e2-a90c-5bfa14c7f245 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.311891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 160/322] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:12.316669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.316955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.317231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.335229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:12.336463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:12.347065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 163/323] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:12.401237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.403154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 161/324] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:12.416223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:12.500921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6f7c9-518e-435c-977a-2cddd5a21982 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:12.534080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:12.534643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.534919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.535187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.536302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:12.537726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:12.539431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 164/325] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:12.545522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.584545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:12.618936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.651094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b47d0-cb8a-4ef2-95c9-fafff7d2ca14 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:12.686508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:12.687123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.687406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.687679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] PATH_INFO: `/v3/groups` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.688342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:12.689774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:12.704918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:12.712318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 162/326] 10.222.0.57 () {60 vars in 1134 bytes} [Thu Sep 15 12:45:12 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:12.712318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.736478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165968a0-9abb-449d-815d-f57ab1f9c009 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:12.777221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:12.778138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.779200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.779633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.796705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:12.801800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:12.802868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:12.803696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 165/327] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:12.809100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.835047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:12.841383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d07b2-a4d3-4123-aa79-4b1b9026b1e5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:12.883001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:12.883644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.884036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.884348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.898864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:12.904795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:12.905681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:12.906627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 163/328] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:12.912108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.925515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676f4e-8e87-455b-bf52-87332af182c4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:12.967818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:12.967818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:12.967818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:12.967818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:12.968241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:12.968722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:12.970078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:12.980080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 166/329] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:12.986590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:12.989752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:13.043675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af669e-2c18-4ac6-854a-858e41dfac2c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:13.099490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:13.100130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:13.100447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:13.101025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:13.102177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:13.102490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:13.103896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:13.120775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 164/330] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:13.126680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:13.127274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b060f-9acf-4316-84d3-bf5c91687610 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:13.155063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:13.169751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:13.170508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:13.170802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:13.171080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] PATH_INFO: `/v3/users/ca50d24d44ea4654994e5c59a6f503e5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:13.172826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] RBAC: Authorizing `identity:get_user(user_id=ca50d24d44ea4654994e5c59a6f503e5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:13.174294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:13.176066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 167/331] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:12 2022] GET /identity/v3/users/ca50d24d44ea4654994e5c59a6f503e5 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:13.181764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:13.254252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb0a2d-70fb-473e-a86a-167d1fc9ff33 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:13.284673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:13.301235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:13.302406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:13.302696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:13.302978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] PATH_INFO: `/v3/users/6a64a7c58cd4496f82e50ea846090eca` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:13.304565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] RBAC: Authorizing `identity:get_user(user_id=6a64a7c58cd4496f82e50ea846090eca)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:13.306159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:13.308134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 165/332] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:13 2022] GET /identity/v3/users/6a64a7c58cd4496f82e50ea846090eca => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:13.314191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:13.329806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099924d-8f6b-4c7e-8e48-3bb3667b4290 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:13.386862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48186520-aa24-47c2-9d3e-3cf728eb769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:13.387602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48186520-aa24-47c2-9d3e-3cf728eb769f None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:13.387934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48186520-aa24-47c2-9d3e-3cf728eb769f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:13.388434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48186520-aa24-47c2-9d3e-3cf728eb769f None admin] PATH_INFO: `/v3/system/users/ca50d24d44ea4654994e5c59a6f503e5/roles/6f21288e54a24472b723fe127cae383e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:13.393101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48186520-aa24-47c2-9d3e-3cf728eb769f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ca50d24d44ea4654994e5c59a6f503e5, role_id=6f21288e54a24472b723fe127cae383e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:13.400120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48186520-aa24-47c2-9d3e-3cf728eb769f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:13.413398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 168/333] 10.222.0.57 () {58 vars in 1334 bytes} [Thu Sep 15 12:45:13 2022] PUT /identity/v3/system/users/ca50d24d44ea4654994e5c59a6f503e5/roles/6f21288e54a24472b723fe127cae383e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:13.443015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7996d3-883f-4478-954d-bbe45069f86f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:13.452681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:13.486558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:13.487199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:13.487520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:13.487863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:13.488707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] RBAC: Adding query filter params (role.id=3901992ddd9a4c30b6faf511834b6e21, user.id=6a64a7c58cd4496f82e50ea846090eca, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:13.489024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:13.490612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff44e0d-c6cc-4057-911c-f858ce58092e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:13.505476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 166/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Sep 15 12:45:13 2022] GET /identity/v3/role_assignments?user.id=6a64a7c58cd4496f82e50ea846090eca&scope.system=all&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:15.388163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911ef92-d367-411e-ba0e-bd519cd7b518 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:15.388663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911ef92-d367-411e-ba0e-bd519cd7b518 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:15.388989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911ef92-d367-411e-ba0e-bd519cd7b518 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:15.390627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 169/335] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:15.396626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f2a7-0844-45e5-91f2-6520a69e3d4c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:15.397152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f2a7-0844-45e5-91f2-6520a69e3d4c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:15.397152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6f2a7-0844-45e5-91f2-6520a69e3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:15.420356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3ad6f2a7-0844-45e5-91f2-6520a69e3d4c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:15.421706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ad6f2a7-0844-45e5-91f2-6520a69e3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:15.509158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 167/336] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:15.669643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4812c5-6f37-459b-a237-c2530927da4b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:15.669959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4812c5-6f37-459b-a237-c2530927da4b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:15.670129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4812c5-6f37-459b-a237-c2530927da4b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:15.671742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 170/337] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:15.676901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6840e20-db19-48db-a62c-74277dcc4703 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:15.677189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6840e20-db19-48db-a62c-74277dcc4703 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:15.677502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6840e20-db19-48db-a62c-74277dcc4703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:15.707071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c6840e20-db19-48db-a62c-74277dcc4703 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:15.708306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c6840e20-db19-48db-a62c-74277dcc4703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:15.798939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 168/338] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:15.988097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4812c5-6f37-459b-a237-c2530927da4b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:16.125924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4812c5-6f37-459b-a237-c2530927da4b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:16.143945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7343e73-093c-4fdf-80ae-5c55ee7b7ff6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.144258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7343e73-093c-4fdf-80ae-5c55ee7b7ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.144539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7343e73-093c-4fdf-80ae-5c55ee7b7ff6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.146044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 169/339] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:16.152735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.153249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.154448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.164467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:16.165367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.165816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.166253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.168496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:16.183392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:16.189060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:16.194594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:16.195490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:16.196301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:16.196908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:16.197966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:16.202270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 171/340] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:16.207866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:16.331439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 170/341] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:16.383548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:16.402069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46091c11-0555-4237-ad80-cf65032a7e1c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:16.478364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:16.478992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.479273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.479545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.480220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:16.480549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:16.481969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:16.497400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 172/342] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:16.504813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:16.570981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13798c-b83a-45cd-8016-7d085f8f4a5a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:16.614217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f4eab-a600-4837-92db-7dd6b63136de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:16.616465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f4eab-a600-4837-92db-7dd6b63136de None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.617029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f4eab-a600-4837-92db-7dd6b63136de None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.617648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f4eab-a600-4837-92db-7dd6b63136de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.625207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:16.640883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f4eab-a600-4837-92db-7dd6b63136de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:16.642751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fd2f4eab-a600-4837-92db-7dd6b63136de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:16.647953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:16.648864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:16.649641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:16.653683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 171/343] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:16.655177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924301-d205-4781-bdec-e381fc70acf8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.655695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924301-d205-4781-bdec-e381fc70acf8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.656178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924301-d205-4781-bdec-e381fc70acf8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.658618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 172/344] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:16.663451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6924301-d205-4781-bdec-e381fc70acf8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:16.717818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea8987-2c87-4842-b663-0470b6ca661a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:16.765668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:16.766287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.766624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.766893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.810468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6924301-d205-4781-bdec-e381fc70acf8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:16.853705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:16.855491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:16.859551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:16.860241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.860656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.861010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.861734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:16.862101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:16.863978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4363b80-955e-4327-b132-fb9b69f5cbc4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:16.876219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 173/345] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:16.877489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:16.877772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:16.878039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:16.905263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:16.906777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:16.929772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:16.973134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 173/346] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:16.974318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:16.980468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:16.998917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 174/347] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:16.999786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.108578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869a3b-8058-45af-b40a-ddd1325c4d17 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.150543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.151342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.151718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.152097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.163548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:17.173943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:17.174933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:17.175749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 174/348] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:17.176446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.203996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:17.207254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccee662-8d62-46fb-af6e-b324ea2cdb3b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.293571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.294742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.295248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.295518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.297511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1f1f2-955a-49bc-b1db-85081ef8b75f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.335644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.336712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.337135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.337547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.338906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:17.340605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:17.344190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 175/349] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:17.345271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.389703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:17.429458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:17.430847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:17.460711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:17.468114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98897376-7c63-4ccd-b6ec-5900af68beb5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.513297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.514419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.514711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.514989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.515670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:17.516034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:17.518255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:17.528662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 176/350] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:17.529763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.582093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 175/351] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:17.583735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.652770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62275-a751-4119-8475-978a5cfa1d1d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.689735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:17.702537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0eb6-4839-48b0-badc-158b98753e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.703203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0eb6-4839-48b0-badc-158b98753e23 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.704477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0eb6-4839-48b0-badc-158b98753e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.704477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0eb6-4839-48b0-badc-158b98753e23 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.709138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ada9d-7d65-4796-8886-6ea78bd2ae5c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.711086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0eb6-4839-48b0-badc-158b98753e23 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:17.712592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0eb6-4839-48b0-badc-158b98753e23 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:17.714256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 177/352] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:17.716728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425993-b869-44ed-8ca9-38da3c300022 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.716728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425993-b869-44ed-8ca9-38da3c300022 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.716728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425993-b869-44ed-8ca9-38da3c300022 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.721735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 178/353] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:17.721735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425993-b869-44ed-8ca9-38da3c300022 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.732255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:17.756302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.756975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.757262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.757518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.770226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:17.777142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:17.778048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:17.778888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 176/354] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:17.779553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.853657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425993-b869-44ed-8ca9-38da3c300022 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.898750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fe47c-d8d6-4d00-b933-c4489471a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.899674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fe47c-d8d6-4d00-b933-c4489471a9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.899953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fe47c-d8d6-4d00-b933-c4489471a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.900224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fe47c-d8d6-4d00-b933-c4489471a9c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.910028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09e138-ddb5-4a85-b70a-72ddd38c5a83 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:17.954049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:17.954696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:17.954974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:17.955252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:17.956412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:17.957883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:17.959681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 177/355] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:17.963116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:17.985971 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fe47c-d8d6-4d00-b933-c4489471a9c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:17.988089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fe47c-d8d6-4d00-b933-c4489471a9c3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:17.993411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.003905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.107119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61fc81-6246-478e-abb5-98f55533c475 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.121909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 179/356] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:18.124314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.124600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.124865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.144354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:18.146346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:18.148858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.149467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.149750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.150019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.152436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=b59720c712524d768e1fc85c3cf651b5) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:18.152436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.152619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.159853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 178/357] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:17 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=0f55ac654fb247249b38e3c701826f02 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.160713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.227720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 180/358] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:18.228602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.269391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.279834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.297260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f863a-10f9-446f-ab53-e1c1e733c56d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.358933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.359574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.360035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.360334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.361058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:18.361375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.362765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9cd6-b41a-4a7f-b1e4-9a4e87c3bc73 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.377158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 179/359] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.377431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cc7e-a6c4-44cc-8a85-1fb010502f70 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.378874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2a23-8fb1-4645-9f48-081c8c0c6904 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.379399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2a23-8fb1-4645-9f48-081c8c0c6904 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.379865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab2a23-8fb1-4645-9f48-081c8c0c6904 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.381574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 180/360] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:18.387527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab2a23-8fb1-4645-9f48-081c8c0c6904 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.432482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924cd18-b76f-4278-b380-f6027f72f17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.433125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924cd18-b76f-4278-b380-f6027f72f17e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.433657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924cd18-b76f-4278-b380-f6027f72f17e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.433942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924cd18-b76f-4278-b380-f6027f72f17e None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.436204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924cd18-b76f-4278-b380-f6027f72f17e None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.437691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924cd18-b76f-4278-b380-f6027f72f17e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.439468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 181/361] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.440466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.440749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.441021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.470809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:18.472979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:18.531839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab2a23-8fb1-4645-9f48-081c8c0c6904 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.559373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 182/362] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:18.561749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.583843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.584585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.584870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.585142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.586363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.588072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.589895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 181/363] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.590941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.609700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.618229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.701404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ee49c-bdc9-4f84-952d-cb4e4851bd5e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.718633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ade269-a2e4-4ca8-8820-d889d07e4aef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.750772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.751902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.752184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.752770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] PATH_INFO: `/v3/users/ca50d24d44ea4654994e5c59a6f503e5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.753943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=ca50d24d44ea4654994e5c59a6f503e5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.755990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.758255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 183/364] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/users/ca50d24d44ea4654994e5c59a6f503e5 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.759384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.772748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.773373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.773656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.773931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.775036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=b59720c712524d768e1fc85c3cf651b5) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:18.775353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.777140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.784001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 182/365] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.784834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:18.914875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.934786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e778f65-1900-4e35-9698-d89af845f1b3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.944309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb709e-d7bc-4d02-b83e-ea433191fa55 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:18.945195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:18.980577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.981993 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.982760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.983273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.986185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:18.986833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:18.987110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:18.987378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:18.988525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:18.989992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:18.992387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 183/366] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:18.993364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.001551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:19.007378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:19.008386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:19.009204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:19.011941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 184/367] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:19.013859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.128952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cae28-afa0-4fb3-b822-80dbda558443 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.163673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023a650-559b-4023-906b-0d671f607c1b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.185014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.185686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.185972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.186678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.187702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] RBAC: Adding query filter params (role.id=6f21288e54a24472b723fe127cae383e, user.id=ca50d24d44ea4654994e5c59a6f503e5, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:19.188018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:19.189493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:19.201545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:19.204400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 184/368] 10.222.0.57 () {56 vars in 1308 bytes} [Thu Sep 15 12:45:18 2022] GET /identity/v3/role_assignments?user.id=ca50d24d44ea4654994e5c59a6f503e5&scope.system=all&role.id=6f21288e54a24472b723fe127cae383e => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:19.205219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.217287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.218062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.218404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.218789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.234826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:19.241723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:19.242717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:19.243512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:19.244661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 185/369] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:19.245926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.300911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:19.403857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79314193-977d-415b-b283-bbde5e94b799 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.414567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68768db9-81ff-4160-976f-8b8b8d3b438b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.445029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.445377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.445630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.445903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.446579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:19.446882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:19.448327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:19.458688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 186/370] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:19.459239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.459597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.460969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.461473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.462151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.463200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=e3991445c4c64256be54a26d63fb7072) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:19.463712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:19.465385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:19.473340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 185/371] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 260 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:19.480746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.572743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:19.591046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692eb01-54da-4d63-91dc-05281eeb980e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.633772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:19.652754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.653426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.653711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.654041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.654671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:19.654983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:19.659150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:19.667397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 187/372] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:19.674539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:19.720513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1837190-eaae-43ec-80ad-54a6abcc4998 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.769361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.769994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.770278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.770559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.844734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2da63-d0ef-494e-87d3-4fe2928f9aca None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:19.851163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:19.853502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:19.917412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:19.917958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:19.918239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:19.918521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:19.924201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:19.965326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:19.970659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 186/373] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:19.976966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:20.090357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:20.092878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:20.101850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27e550-4296-4610-acfa-950639f67647 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:20.142254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:20.142901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.143184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.143463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:20.144690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:20.146127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:20.148136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 187/374] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:20.154466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:20.217370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:20.289426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 188/375] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:45:20.295809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:20.376529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:20.377047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683e0d7-3c20-4d13-ba6d-adf31fe90311 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:20.445317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:20.446349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.446834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.447310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:20.449793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:20.452104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:20.460396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 188/376] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:20 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:20.466158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:20.472435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b1ba-c4ea-4a9f-89a7-db8b7e6e8ea5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:20.544123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:20.545166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.545658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.546130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] PATH_INFO: `/v3/projects/53ac1f185c9948448574c644790cea74` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:20.550444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] RBAC: Authorizing `identity:get_project(project_id=53ac1f185c9948448574c644790cea74)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:20.552142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:20.556742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 189/377] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:20 2022] GET /identity/v3/projects/53ac1f185c9948448574c644790cea74 => generated 344 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:20.560546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:20.640871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:20.647338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0b8b-923b-425c-a007-b0e004e6c94b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:20.692962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:20.693942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.694442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.695017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:20.695937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=e3991445c4c64256be54a26d63fb7072) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:20.696498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:20.698131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:20.705549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 189/378] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:20 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=0f55ac654fb247249b38e3c701826f02 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:20.770587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407231a-9a69-4835-bc59-d214f5b2c22c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:20.801731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:20.822808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904892df-0666-48bb-8bdf-1fd1ad619391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:20.823480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904892df-0666-48bb-8bdf-1fd1ad619391 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.823762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904892df-0666-48bb-8bdf-1fd1ad619391 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.824033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904892df-0666-48bb-8bdf-1fd1ad619391 None admin] PATH_INFO: `/v3/groups/ed17f30b1643424ca52150307d00a19f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:20.837093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904892df-0666-48bb-8bdf-1fd1ad619391 None admin] RBAC: Authorizing `identity:get_group(group_id=ed17f30b1643424ca52150307d00a19f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:20.838933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904892df-0666-48bb-8bdf-1fd1ad619391 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:20.840970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 190/379] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:20 2022] GET /identity/v3/groups/ed17f30b1643424ca52150307d00a19f => generated 229 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:20.847443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:20.988634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee673a-a619-410c-b869-e1a4e6d6067e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.989404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee673a-a619-410c-b869-e1a4e6d6067e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.989892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee673a-a619-410c-b869-e1a4e6d6067e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:20.991787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 191/380] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:20.997916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:20.998216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:20.998492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:21.015325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ec1-2ce3-4755-9a64-bfdba7af182e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:21.022521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:21.023662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:21.062200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:21.063117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:21.063418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:21.063722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:21.064466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] RBAC: Adding query filter params (group.id=ed17f30b1643424ca52150307d00a19f, role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=53ac1f185c9948448574c644790cea74) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:21.064784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:21.066231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:21.073842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 190/381] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:20 2022] GET /identity/v3/role_assignments?group.id=ed17f30b1643424ca52150307d00a19f&scope.project.id=53ac1f185c9948448574c644790cea74&role.id=13d4313b2fe74b3688999f3077afce5d => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:21.124028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 192/382] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:21.602612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:21.656050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:21.777631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1170e2b-7ed1-4feb-a972-432e2c0069a7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:21.830048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dec214-7d95-4cee-adcb-a157cf01386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:21.830708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dec214-7d95-4cee-adcb-a157cf01386c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:21.830995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dec214-7d95-4cee-adcb-a157cf01386c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:21.831273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dec214-7d95-4cee-adcb-a157cf01386c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:21.847014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dec214-7d95-4cee-adcb-a157cf01386c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a9dec214-7d95-4cee-adcb-a157cf01386c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:21.852516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:21.853472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:21.854262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 191/383] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:21.862384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:21.922811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ac77-6ce9-464d-b089-13e8851b63d1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:21.923137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ac77-6ce9-464d-b089-13e8851b63d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:21.923381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ac77-6ce9-464d-b089-13e8851b63d1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:21.925082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 192/384] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:21.936387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:21.937036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:21.937528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:21.975247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:21.977024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:22.082422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78a7b8-3680-4b83-aced-e71c9b3fa0cb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:22.112253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:22.154016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 193/385] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:22.162858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:22.163494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.163776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.164049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.164826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:22.165137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:22.166589 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5491de7-a976-481f-b8ff-5ea3898124ca None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:22.176655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 193/386] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:22.185964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:22.218887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:22.279503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03463031-3df6-4aaa-81e4-f17588eb7fab None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.280136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03463031-3df6-4aaa-81e4-f17588eb7fab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.280693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03463031-3df6-4aaa-81e4-f17588eb7fab None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.282565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 194/387] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:22.288593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.289175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.289667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.311294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:22.313006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:22.315812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6e270-619e-4776-953a-8bc5914c26c1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:22.371575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:22.373180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.373689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.374185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.375582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:22.377279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:22.379439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 194/388] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:22 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:22.384905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:22.435071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 195/389] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:22.513625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:22.532012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe75b22-de03-4130-b736-a1bc8160dd63 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:22.568816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:22.569448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.569724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.569992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.571111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:22.572581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:22.574478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 195/390] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:22 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:22.583624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:22.634642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:22.768973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14976a-a3ed-4333-bf4b-59e145556cf7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:22.775922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:22.842276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48736cc5-f81b-47cf-bb10-817e3edfef76 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:22.845629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:22.846610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.847115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.847599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5/groups/bf7f689f97d34f77b16fe32660f654fd/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.850010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] RBAC: Authorizing `identity:create_grant(project_id=b59720c712524d768e1fc85c3cf651b5, group_id=bf7f689f97d34f77b16fe32660f654fd, role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:22.851672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:22.872998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 196/391] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Sep 15 12:45:22 2022] PUT /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5/groups/bf7f689f97d34f77b16fe32660f654fd/roles/0f55ac654fb247249b38e3c701826f02 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:22.873969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:22.879976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:22.880660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:22.880943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:22.881218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:22.896821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:22.901729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:22.902608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:22.903456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:22.909089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 196/392] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:22.915142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:22.945243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:23.058226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09681e00-6d88-4e8f-ae7a-c11f25af218f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.092986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d86a63-0a21-4b05-928d-b94cc01d373b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.098147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.099055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.099535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.100012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.112349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:23.117964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:23.118843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:23.119627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:23.122443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 197/393] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:23.128158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.170480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:23.181645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.182269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.182550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.182820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.183474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:23.183778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:23.189272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:23.202254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 197/394] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:23.208066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.277441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735d55-8fe7-4a1b-a023-a42223c49076 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.306519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:23.334852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9830d5c-e458-470e-a429-4d5f3bf72b82 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.374114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.374739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.375026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.375287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.375783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.375970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:23.376257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:23.377048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.377523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.377644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:23.378100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.379636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:23.381276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:23.383069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 198/395] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:23 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:23.389428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.403225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 198/396] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:23.409551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.588730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:23.598832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13a6f3-d5cd-44ee-8243-08c8aec3646f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.638456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a250bb-6e61-45e6-8012-e7a62a5f7a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.639540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a250bb-6e61-45e6-8012-e7a62a5f7a9e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.640021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a250bb-6e61-45e6-8012-e7a62a5f7a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.640540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a250bb-6e61-45e6-8012-e7a62a5f7a9e None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.642005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369c070-0070-416f-aae7-6f055a853707 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.642144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a250bb-6e61-45e6-8012-e7a62a5f7a9e None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:23.643921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a250bb-6e61-45e6-8012-e7a62a5f7a9e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:23.645926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 199/397] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:23 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:23.647274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a7cb-eccd-4c71-8936-fbd4d603b13b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.647753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a7cb-eccd-4c71-8936-fbd4d603b13b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.648285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a7cb-eccd-4c71-8936-fbd4d603b13b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.649923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 200/398] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:23.655946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1a7cb-eccd-4c71-8936-fbd4d603b13b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.687487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2735e-bbc9-4c60-9303-09b03175c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.688193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2735e-bbc9-4c60-9303-09b03175c0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.688560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2735e-bbc9-4c60-9303-09b03175c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.688892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2735e-bbc9-4c60-9303-09b03175c0f9 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.690152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2735e-bbc9-4c60-9303-09b03175c0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:23.691696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2735e-bbc9-4c60-9303-09b03175c0f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:23.693980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 199/399] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:23 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:23.699921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.700892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.701330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.720477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:23.721782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:23.730122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:23.792978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1a7cb-eccd-4c71-8936-fbd4d603b13b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:23.804308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 200/400] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:23.804308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.878077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:23.878656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:23.878907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:23.879176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:23.881940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:23.883386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:23.886983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 201/401] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:23 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:23.895099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:23.939358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:23.996470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f31c-3ff1-46c9-b0af-0ed995cf5dd7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.040705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95d6f5-d8e6-43f1-8c3e-3e8a11874edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.041378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95d6f5-d8e6-43f1-8c3e-3e8a11874edc None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.041656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95d6f5-d8e6-43f1-8c3e-3e8a11874edc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.042033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95d6f5-d8e6-43f1-8c3e-3e8a11874edc None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5/groups/bf7f689f97d34f77b16fe32660f654fd/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.044145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95d6f5-d8e6-43f1-8c3e-3e8a11874edc None admin] RBAC: Authorizing `identity:create_grant(project_id=b59720c712524d768e1fc85c3cf651b5, group_id=bf7f689f97d34f77b16fe32660f654fd, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:24.046343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95d6f5-d8e6-43f1-8c3e-3e8a11874edc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:24.052632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60840091-2efb-42c6-9eff-f4b520e2f372 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.067487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 201/402] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Sep 15 12:45:23 2022] PUT /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5/groups/bf7f689f97d34f77b16fe32660f654fd/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:24.069326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dc41a-7fcf-4155-bb82-70b0f415311d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.069651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dc41a-7fcf-4155-bb82-70b0f415311d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.069985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5dc41a-7fcf-4155-bb82-70b0f415311d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.071717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 202/403] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:24.078134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.078761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.079235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.087461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:24.105000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:24.106818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:24.124185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.124976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.125357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.125636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072/groups/bf7f689f97d34f77b16fe32660f654fd/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.127771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3991445c4c64256be54a26d63fb7072, group_id=bf7f689f97d34f77b16fe32660f654fd, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:24.129360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:24.148032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 202/404] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Sep 15 12:45:23 2022] PUT /identity/v3/projects/e3991445c4c64256be54a26d63fb7072/groups/bf7f689f97d34f77b16fe32660f654fd/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:24.149277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.196413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 203/405] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:24.257447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:24.281085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1538b-011c-45c0-8464-69e73e5522d1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.331548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.332220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.332540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.332817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.346172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:24.352189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:24.353335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:24.354287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 203/406] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:24.360115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.480331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73abf7-73ad-4b50-8136-5ce6de2ee9c8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.513833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:24.519359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.519982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.520300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.520627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.521297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:24.521606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:24.522977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:24.536222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.543286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 204/407] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:24.548224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.704068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52348c3-8d4c-4319-8e12-6eec14130fb1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.706325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999537f-bf88-41dd-9ee4-dae53807b20c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.751957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.752540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.752820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.753094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.754204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:24.755640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:24.757404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 205/408] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:24 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:24.757529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.758118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.758404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.758677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.763528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.782282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:24.787630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:24.788530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:24.789310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 204/409] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:24.797011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.802673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:24.873674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:24.912753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47a89d-da0b-4fe9-85fa-5ff2b79b77ae None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:24.948476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:24.949114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:24.949397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:24.949678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:24.951467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:24.953012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:24.955129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 206/410] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:24 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:24.960768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:24.990012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3a4b6-a01b-466f-a949-a3a5ae32bf83 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:25.058808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:25.059991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.060292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.060559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.061227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:25.061525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:25.063117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:25.068443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068b6d7-3fde-4427-8eaf-fdccacff86df None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:25.080104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 205/411] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:25.089919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:25.104311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:25.111520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:25.112228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.112552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.112832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072/groups/bf7f689f97d34f77b16fe32660f654fd/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.115059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] RBAC: Authorizing `identity:create_grant(project_id=e3991445c4c64256be54a26d63fb7072, group_id=bf7f689f97d34f77b16fe32660f654fd, role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:25.116596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:25.135119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 207/412] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Sep 15 12:45:24 2022] PUT /identity/v3/projects/e3991445c4c64256be54a26d63fb7072/groups/bf7f689f97d34f77b16fe32660f654fd/roles/0f55ac654fb247249b38e3c701826f02 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:45:25.190369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:25.243151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29aaf5e-359c-426f-8830-ee484996c495 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:25.291563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:25.292219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.292603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.292885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] PATH_INFO: `/v3/groups/ed17f30b1643424ca52150307d00a19f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.297091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] RBAC: Authorizing `identity:get_group(group_id=ed17f30b1643424ca52150307d00a19f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:25.298656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:25.300468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 206/413] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:25 2022] GET /identity/v3/groups/ed17f30b1643424ca52150307d00a19f => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:25.306267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:25.438740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25d888-8e2d-4e62-bf02-7a500ba15b2a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:25.474698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182446a-df83-441d-a086-4ea71c7c505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:25.475337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182446a-df83-441d-a086-4ea71c7c505e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.475620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182446a-df83-441d-a086-4ea71c7c505e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.475892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182446a-df83-441d-a086-4ea71c7c505e None admin] PATH_INFO: `/v3/projects/53ac1f185c9948448574c644790cea74` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.477062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182446a-df83-441d-a086-4ea71c7c505e None admin] RBAC: Authorizing `identity:get_project(project_id=53ac1f185c9948448574c644790cea74)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:25.478510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182446a-df83-441d-a086-4ea71c7c505e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:25.484510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 208/414] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:25 2022] GET /identity/v3/projects/53ac1f185c9948448574c644790cea74 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:25.486905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:25.524469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0939b7d-917b-4447-8f93-93e8580d911f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.524792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0939b7d-917b-4447-8f93-93e8580d911f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.525029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0939b7d-917b-4447-8f93-93e8580d911f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.526604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 209/415] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:25.539298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.539587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.539867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.569732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:25.570934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:25.655311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e7249-6cfc-4a30-ab57-1b9f6cd9e449 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:25.672361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:25.702797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 210/416] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:25.714199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:25.714862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:25.715143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:25.715417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] PATH_INFO: `/v3/projects/53ac1f185c9948448574c644790cea74/groups/ed17f30b1643424ca52150307d00a19f/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:25.718046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] RBAC: Authorizing `identity:create_grant(project_id=53ac1f185c9948448574c644790cea74, group_id=ed17f30b1643424ca52150307d00a19f, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:25.719490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:25.725167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:25.740472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 207/417] 10.222.0.57 () {58 vars in 1442 bytes} [Thu Sep 15 12:45:25 2022] PUT /identity/v3/projects/53ac1f185c9948448574c644790cea74/groups/ed17f30b1643424ca52150307d00a19f/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:25.998636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:26.134467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648d9bd-cf25-4928-8b6d-cf6118a99dd3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:26.188911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:26.189605 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:26.189888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:26.190161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:26.209124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:26.214030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:26.214916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:26.216314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 211/418] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:26.221666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:26.372729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b7be1-9aec-402c-ab8a-fbf948daa0ee None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:26.421545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:26.422630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:26.423120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:26.423596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:26.424544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:26.425062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:26.426717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:26.440302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 208/419] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:26.445688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:26.525502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:26.594246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4b37f-a94e-49fb-b9d6-a2d7e1e6c0c1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:26.657655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178297b0-71d6-49de-879d-997071eb2ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:26.658194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178297b0-71d6-49de-879d-997071eb2ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:26.658480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178297b0-71d6-49de-879d-997071eb2ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:26.658756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178297b0-71d6-49de-879d-997071eb2ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:26.756862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178297b0-71d6-49de-879d-997071eb2ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:26.758328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178297b0-71d6-49de-879d-997071eb2ef2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:26.830076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:26.904372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 212/420] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:26.905773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:26.934991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10006676-8086-47be-90ac-82c30fb58f4d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:26.935302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10006676-8086-47be-90ac-82c30fb58f4d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:26.935708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10006676-8086-47be-90ac-82c30fb58f4d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:26.937607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 213/421] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:26.959759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:26.960040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:26.960357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:26.979342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:26.980573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:26.985777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:27.053773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5c8de-081e-4055-bd93-2ee30ea4391e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:27.065669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 214/422] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:27.111151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94c4e5-8fdf-43c6-bd9f-9abc66d8c2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:27.111850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c4e5-8fdf-43c6-bd9f-9abc66d8c2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.112139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c4e5-8fdf-43c6-bd9f-9abc66d8c2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.112444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94c4e5-8fdf-43c6-bd9f-9abc66d8c2a8 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.114773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94c4e5-8fdf-43c6-bd9f-9abc66d8c2a8 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:27.116303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94c4e5-8fdf-43c6-bd9f-9abc66d8c2a8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:27.117946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 209/423] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:26 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:27.124252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.144216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94c9b7-e8fc-46b1-84bb-de33a8d151a3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.144532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94c9b7-e8fc-46b1-84bb-de33a8d151a3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.144809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94c9b7-e8fc-46b1-84bb-de33a8d151a3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.146257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 210/424] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:27.156505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.156784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.157053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.192390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:27.193554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:27.229598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:27.270258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e5db-285d-45bd-ae1c-ea06496b7246 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:27.310067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:27.310736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.311020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.311296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.312620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:27.314420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:27.316319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 215/425] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:27.322180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.347147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 211/426] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:27.427860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84f12-0202-4f54-83a6-f9631d9f6e58 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:27.440826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.487221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:27.487844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.488129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.488431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.489167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=b59720c712524d768e1fc85c3cf651b5) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:27.489481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:27.490925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:27.512117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 216/427] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=0f55ac654fb247249b38e3c701826f02 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:27.528065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:27.580612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d2a4a-ce22-402a-91d8-3c91bef68f6f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:27.628951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:27.629594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.629876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.630149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.646001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:27.651053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:27.651913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:27.652756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 212/428] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:27.658562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.674204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.682481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:27.830045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3790974-0b0f-4db5-bca4-8c7dc146abf0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:27.832595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d79c9d-7614-4744-8024-48dfff05a52c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:27.857097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:27.871404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:27.872757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.873332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.873810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.874655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:27.875162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:27.876748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:27.880583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:27.881443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:27.881897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:27.882360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:27.891822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 217/429] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:27.893576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.900914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:27.906273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:27.907159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:45:27.907937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:27.915408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 213/430] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:27.916879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:27.978159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.014147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bbd7b-39c9-4947-8121-f58a8d567743 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.035179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9831a-0ebf-4273-a629-90605e65fd39 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.050895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed056a-3c45-4618-befd-3f0430ef2365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.051522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed056a-3c45-4618-befd-3f0430ef2365 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.051807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed056a-3c45-4618-befd-3f0430ef2365 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.052420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed056a-3c45-4618-befd-3f0430ef2365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.072651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.073265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.073543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.073814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.074733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:28.074883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.076314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd4184-46cc-4e2e-a147-e5510e759c15 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.089527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 214/431] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:28.091630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bc52c-43fd-4c34-8b3d-67975e77e7fa None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.091922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bc52c-43fd-4c34-8b3d-67975e77e7fa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.092197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bc52c-43fd-4c34-8b3d-67975e77e7fa None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.093612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 215/432] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:28.094339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bc52c-43fd-4c34-8b3d-67975e77e7fa None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.131675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.133379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed056a-3c45-4618-befd-3f0430ef2365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.135083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed056a-3c45-4618-befd-3f0430ef2365 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.204285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bc52c-43fd-4c34-8b3d-67975e77e7fa None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.209417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.248831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 218/433] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:28.249429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.249521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.249702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.249773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.249773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.249887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.249887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.266064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:28.267451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:28.318989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.320461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.348384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 219/434] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:28.348384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.372601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.449280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 216/435] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:28.451343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.491820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e1a4f-a01e-4d2d-8a59-232f6a0dee61 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.493412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.530255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b43fea-d42b-48c8-9336-525b8fc75401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.530890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43fea-d42b-48c8-9336-525b8fc75401 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.531171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43fea-d42b-48c8-9336-525b8fc75401 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.531445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b43fea-d42b-48c8-9336-525b8fc75401 None admin] PATH_INFO: `/v3/projects/b59720c712524d768e1fc85c3cf651b5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.532622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43fea-d42b-48c8-9336-525b8fc75401 None admin] RBAC: Authorizing `identity:get_project(project_id=b59720c712524d768e1fc85c3cf651b5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.534053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b43fea-d42b-48c8-9336-525b8fc75401 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.535830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 220/436] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/projects/b59720c712524d768e1fc85c3cf651b5 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:28.536943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dba70-dd11-433e-ab97-e775e31d6b37 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.537228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dba70-dd11-433e-ab97-e775e31d6b37 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.537496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dba70-dd11-433e-ab97-e775e31d6b37 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.538804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 221/437] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:28.541241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dba70-dd11-433e-ab97-e775e31d6b37 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a5e76-c7e6-4ca5-b6a3-ed2842bb014d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.601578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97a0b6-1584-4a44-902d-47c465b2e069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.602129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97a0b6-1584-4a44-902d-47c465b2e069 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.602409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97a0b6-1584-4a44-902d-47c465b2e069 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.602681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97a0b6-1584-4a44-902d-47c465b2e069 None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.604088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97a0b6-1584-4a44-902d-47c465b2e069 None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.605561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97a0b6-1584-4a44-902d-47c465b2e069 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.607667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 217/438] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:28.613594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.614141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.614674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.620763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.633663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:28.634861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:28.646018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dba70-dd11-433e-ab97-e775e31d6b37 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.679748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.680697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.681077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.681431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.682561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.683977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.685621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 222/439] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:28.686367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.707269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 218/440] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:28.708116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.725868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.787803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03964f1f-c241-460c-8589-2e7deaf63f6f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.817555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276fecf-283c-4ef8-8b79-55b607e0f690 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.823196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.823826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.824109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.824418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.826430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.827863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.829628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 223/441] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:28.833130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.850841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.857714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.858428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.858705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.858976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.869097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:28.874577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:28.875487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 219/442] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:28.876933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:28.951336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b48a94-6406-429f-851b-0ed20ca2b9e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.959745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:28.982723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f771f-226a-4515-8f72-ee82c063b435 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:28.988052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:28.988689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:28.988973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:28.989249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:28.989972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=b59720c712524d768e1fc85c3cf651b5) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:28.990284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:28.991706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:28.997932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 224/443] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=b59720c712524d768e1fc85c3cf651b5&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:28.998778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.021973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.022599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.022879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.023152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.023909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=3901992ddd9a4c30b6faf511834b6e21, scope.project.id=e3991445c4c64256be54a26d63fb7072) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:29.024218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.025782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.032959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 220/444] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=3901992ddd9a4c30b6faf511834b6e21 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:29.033816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.118409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.143415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8719a-9e82-4180-89e5-09b020ad37ab None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.175147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c5b9d-2776-4980-8e28-53661c45b082 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.188202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.188871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.189156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.189434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.190120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:29.190427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.192104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.204320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 225/445] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:29.210577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.231174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.232476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.232768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.233040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.249886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:29.253196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:29.257588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:29.258490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:29.259278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 221/446] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:29.265729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.350517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e92f2-db1e-4659-9957-3b9f04daeeb5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.390131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213638fa-495e-40aa-885d-fdad0ffe7b64 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.393608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.394231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.394513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.394820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.410221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.433998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.436456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.436456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.436456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.436781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:29.437087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.438475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.449165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 222/447] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:29.454558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.484879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.486610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.520230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.575918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92838162-d5de-4b99-8705-270a63b96222 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.611089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 226/448] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:45:29.616316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.616881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.616997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.617196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.617466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.671227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.698239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.699695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.745105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06121d7-ccd7-40dc-a657-08e1ed24e89c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.781959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.787341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.787973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.788258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.788586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] PATH_INFO: `/v3/projects/e3991445c4c64256be54a26d63fb7072` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.789699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] RBAC: Authorizing `identity:get_project(project_id=e3991445c4c64256be54a26d63fb7072)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.791206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.792859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 227/449] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/projects/e3991445c4c64256be54a26d63fb7072 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:29.798972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.812226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 223/450] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:29.817638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.896035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:29.899064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921982d3-b85d-421b-9c55-6362875d28ab None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.916836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b0b58-ec87-4e2d-b6da-8d2dbe8fc95d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:29.933529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.934179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.935031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.935492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] PATH_INFO: `/v3/groups/bf7f689f97d34f77b16fe32660f654fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.936665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] RBAC: Authorizing `identity:get_group(group_id=bf7f689f97d34f77b16fe32660f654fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.938094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.939660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 228/451] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/groups/bf7f689f97d34f77b16fe32660f654fd => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:29.945398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:29.951151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:29.951769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:29.952047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:29.952344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] PATH_INFO: `/v3/projects/53ac1f185c9948448574c644790cea74` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:29.953464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] RBAC: Authorizing `identity:get_project(project_id=53ac1f185c9948448574c644790cea74)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:29.954945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:29.956739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 224/452] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/projects/53ac1f185c9948448574c644790cea74 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:29.961853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:30.014024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:30.047914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051da3e-cfbb-4cbd-824c-e96e057291f5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:30.058925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab6771-3a96-46dc-abc1-b2cdfc22b23a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:30.080977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:30.081584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:30.081863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:30.082135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:30.082849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] RBAC: Adding query filter params (group.id=bf7f689f97d34f77b16fe32660f654fd, role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=e3991445c4c64256be54a26d63fb7072) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:30.083161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:30.084622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:30.090340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 229/453] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/role_assignments?group.id=bf7f689f97d34f77b16fe32660f654fd&scope.project.id=e3991445c4c64256be54a26d63fb7072&role.id=0f55ac654fb247249b38e3c701826f02 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:30.095189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc2734-92e3-455a-b865-4f375385d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:30.095901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2734-92e3-455a-b865-4f375385d5df None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:30.096189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2734-92e3-455a-b865-4f375385d5df None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:30.096495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2734-92e3-455a-b865-4f375385d5df None admin] PATH_INFO: `/v3/groups/ed17f30b1643424ca52150307d00a19f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:30.097709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc2734-92e3-455a-b865-4f375385d5df None admin] RBAC: Authorizing `identity:get_group(group_id=ed17f30b1643424ca52150307d00a19f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:30.099193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc2734-92e3-455a-b865-4f375385d5df None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:30.100877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 225/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu Sep 15 12:45:29 2022] GET /identity/v3/groups/ed17f30b1643424ca52150307d00a19f => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:30.107567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:30.214799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967a05-0a30-4ae5-ba81-514228a95c1f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:30.245523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:30.251650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:30.252265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:30.252580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:30.252857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:30.253596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] RBAC: Adding query filter params (group.id=ed17f30b1643424ca52150307d00a19f, role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=53ac1f185c9948448574c644790cea74) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:30.254169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:30.255337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe43a1e-00d0-4e2e-bba2-615e0276a630 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:30.262568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 230/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu Sep 15 12:45:30 2022] GET /identity/v3/role_assignments?group.id=ed17f30b1643424ca52150307d00a19f&scope.project.id=53ac1f185c9948448574c644790cea74&role.id=13d4313b2fe74b3688999f3077afce5d => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:32.897992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdada5b-229f-4c18-a713-c5066e13e258 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:32.898996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdada5b-229f-4c18-a713-c5066e13e258 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:32.899474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdada5b-229f-4c18-a713-c5066e13e258 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:32.901269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 226/456] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:32.907233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2da2e7-7063-425e-bc0f-c36bacebdf53 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:32.908051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2da2e7-7063-425e-bc0f-c36bacebdf53 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:32.908566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2da2e7-7063-425e-bc0f-c36bacebdf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:32.937291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5b2da2e7-7063-425e-bc0f-c36bacebdf53 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:32.938841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5b2da2e7-7063-425e-bc0f-c36bacebdf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:33.030816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 231/457] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:33.224082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4fd4-b898-4912-b2ea-a05710fe35da None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.224778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4fd4-b898-4912-b2ea-a05710fe35da None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.225258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4fd4-b898-4912-b2ea-a05710fe35da None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.226781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 227/458] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:33.241926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.242515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.242984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.262934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a68d-27c5-44b7-aab8-ebbe7fbacffd None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.263730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a68d-27c5-44b7-aab8-ebbe7fbacffd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.264233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a68d-27c5-44b7-aab8-ebbe7fbacffd None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.266059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 228/459] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:33.270358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:33.271603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.271883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.272185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:33.272371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.297548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:33.298764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:33.321495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:33.388482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 232/460] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:33.389704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:33.442846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 229/461] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:33.462008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:33.505138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718d73-6428-418c-b0ba-587fad5a54b4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:33.541220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:33.542082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.542505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.542943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.554360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:33.556028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:33.571590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:33.572722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:33.573761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:33.573761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:33.573761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:33.580136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 233/462] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:33.581673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:33.700359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548993f-6fc5-4c15-b610-6d3271c27c47 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:33.735164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:33.736028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.736516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.736951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.737732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:33.738174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:33.739721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:33.759580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:33.766403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 230/463] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:33.781546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:33.786681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:33.882499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cd5b-f0ca-4a20-8b12-cd1a678e2519 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:33.893316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:33.908986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac32077-0a37-442c-861c-cde3c8c73361 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:33.920849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:33.921583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.921935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.922279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.933447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:33.935543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:33.947158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:33.947961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.948457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.948869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:33.949757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:33.950789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:33.951780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:33.951780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:33.951780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:33.951780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 234/464] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:33.952130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:33.965108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:33.967709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dd7f6e29-4de9-4415-9a21-c81fc03a79bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:33.985194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:33.986207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:33.987013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:33.987013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:33.987013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:33.988666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 231/465] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:33.990160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e4f43-2582-4dae-aac4-816aca33fa73 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:33.990588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e4f43-2582-4dae-aac4-816aca33fa73 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:33.990999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e4f43-2582-4dae-aac4-816aca33fa73 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:33.992379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 232/466] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:33.993392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e4f43-2582-4dae-aac4-816aca33fa73 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.060914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:34.070547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ec2ca-917f-4cd2-9937-a7bc6c2d5f2f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:34.110104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:34.111083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.111576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.111998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.112836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:34.114181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e4f43-2582-4dae-aac4-816aca33fa73 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:34.114588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:34.140674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 235/467] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:33 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:34.141505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.162566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:34.167506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:34.169202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.169674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.170374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.171193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:34.171818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:34.174242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524e40d-6e83-4eb4-8cd5-a5766e6e4bf3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:34.192632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 233/468] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:34.194269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.194703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.195125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.220762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:34.223032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:34.270593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d807a1e-72a0-4076-a0b1-1f5197cae13c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:34.313654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:34.313848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 234/469] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:34.320547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.321770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.322215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.322655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.323477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:34.323938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:34.326324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:34.350313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 236/470] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:34.357000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.391398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:34.454790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29bf9c2-8077-4fe2-abc4-4bf727c3ed63 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:34.456055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:34.485317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed16282-bb75-4ba6-a498-4b14bd55bcf3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:34.495059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b720c-a436-42f4-bf23-f6e210a6124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:34.496054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b720c-a436-42f4-bf23-f6e210a6124c None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.496740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b720c-a436-42f4-bf23-f6e210a6124c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.497261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b720c-a436-42f4-bf23-f6e210a6124c None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.498273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b720c-a436-42f4-bf23-f6e210a6124c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:34.500030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b720c-a436-42f4-bf23-f6e210a6124c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:34.524469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 235/471] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:34 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:34.526016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001872d8-f335-4de6-af5b-43cd4dc2d4d9 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.526489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001872d8-f335-4de6-af5b-43cd4dc2d4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.526968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001872d8-f335-4de6-af5b-43cd4dc2d4d9 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.529606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 236/472] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:34.529749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:34.530570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.530832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.531185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.533886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:34.537760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:34.541877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.542167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.542442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.571562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 237/473] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:34 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:34.572216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:34.573730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:34.644714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.664464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 237/474] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:34.834161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc26ef-9034-4d4c-b558-2c4bab27b1a8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:34.867781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:34.875951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:34.876793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:34.877075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:34.877513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:34.891034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:34.892788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:34.908019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:34.908976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:34.909828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:34.909828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:34.909828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:34.909828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 238/475] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:34.915573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.948247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:34.951229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:35.058351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436b04f-a812-41d8-87f5-6d2f7a2e20cd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:35.067942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5202a-1f2a-4757-88e5-9593a9a2b212 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:35.115585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:35.116418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:35.116712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:35.116989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:35.117644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:35.117952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:35.118262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:35.119111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:35.119339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:35.119673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:35.120083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:35.121004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:35.123392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:35.131442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 12:45:35.131442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 12:45:35.131442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [parameters: {'id': 'b012ad29ef694cb39a613e0163f0e0c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 12:45:35.131442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 12:45:35.133353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 238/476] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:35.142369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 15 12:45:35.153928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return ref.to_dict() Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:45:35.154848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 12:45:35.155721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 12:45:35.156580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:45:35.157405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:45:35.158203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': 'b012ad29ef694cb39a613e0163f0e0c7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:35.158942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 12:45:35.159700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:35.166239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 239/477] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Sep 15 12:45:34 2022] POST /identity/v3/roles => generated 144 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:35.166239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:35.216221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:35.281686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22443e-5088-4869-8146-d631a2d9192f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:35.319863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:35.320683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:35.320980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:35.321265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:35.333004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:35.338966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:35.339851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:35.340947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 240/478] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:45:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:35.346405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:35.417459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:35.433139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06a07-cae0-47b7-aa98-48d3084690af None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:35.465081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb10ea-2206-46af-bb82-2fed314314c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:35.475288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:35.504756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6f06-3fb8-4305-8396-d97f88e5551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:35.505374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:35.505975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6f06-3fb8-4305-8396-d97f88e5551d None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:35.506080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:35.506263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:35.506545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:35.506950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6f06-3fb8-4305-8396-d97f88e5551d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:35.507215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:35.507526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:35.507785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6f06-3fb8-4305-8396-d97f88e5551d None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:35.508718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b6f06-3fb8-4305-8396-d97f88e5551d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:35.508943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14214a9-f95f-4e42-8aae-0bcfd46095c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:35.510688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b6f06-3fb8-4305-8396-d97f88e5551d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:35.518566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 241/479] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:35.556718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 239/480] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:35 2022] POST /identity/v3/users => generated 249 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:45:37.021811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0acd66-30bc-40c6-8d87-4ed105fce76b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.022475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0acd66-30bc-40c6-8d87-4ed105fce76b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.022475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0acd66-30bc-40c6-8d87-4ed105fce76b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.023879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 242/481] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:37.030169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70216910-3241-452b-9f02-6446c44998b8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.030637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70216910-3241-452b-9f02-6446c44998b8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.030711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70216910-3241-452b-9f02-6446c44998b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.054180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-70216910-3241-452b-9f02-6446c44998b8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:37.055410 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-70216910-3241-452b-9f02-6446c44998b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:37.094473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb7135-fc6c-433f-b509-d014b19965c3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.094473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb7135-fc6c-433f-b509-d014b19965c3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.094473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb7135-fc6c-433f-b509-d014b19965c3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.096204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 243/482] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:37.101973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.102268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.102552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.135777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:37.137296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:37.186727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 240/483] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:37.245889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 244/484] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:37.581726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70216910-3241-452b-9f02-6446c44998b8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:37.600074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:37.719585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:37.748377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:37.749904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70216910-3241-452b-9f02-6446c44998b8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:37.754288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340770f2-e566-4029-9730-f38bdb41a0bd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:37.790221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0325e-f4fc-4c95-b55c-598f3c87ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:37.790823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0325e-f4fc-4c95-b55c-598f3c87ded1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.791112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0325e-f4fc-4c95-b55c-598f3c87ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.791388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0325e-f4fc-4c95-b55c-598f3c87ded1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.797797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75b6cb-3799-4dea-96dc-ebcbcd863e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:37.798776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75b6cb-3799-4dea-96dc-ebcbcd863e72 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.799379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75b6cb-3799-4dea-96dc-ebcbcd863e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.799845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75b6cb-3799-4dea-96dc-ebcbcd863e72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.802352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0325e-f4fc-4c95-b55c-598f3c87ded1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-78c0325e-f4fc-4c95-b55c-598f3c87ded1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:37.807275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:37.808142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:37.809476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 241/485] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:37.814597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75b6cb-3799-4dea-96dc-ebcbcd863e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:37.815431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190d7f1-f3d1-40fe-bfbb-b7e6fdbbe757 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.815724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190d7f1-f3d1-40fe-bfbb-b7e6fdbbe757 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.816001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190d7f1-f3d1-40fe-bfbb-b7e6fdbbe757 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.817546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 242/486] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:37.821737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190d7f1-f3d1-40fe-bfbb-b7e6fdbbe757 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1b75b6cb-3799-4dea-96dc-ebcbcd863e72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:37.823551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:37.824471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:37.826453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 245/487] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:37.829132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:37.829964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:37.830433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:37.852796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:37.854335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:37.944609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 246/488] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:37.945718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:37.953016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190d7f1-f3d1-40fe-bfbb-b7e6fdbbe757 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.023545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.024181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.024488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.024758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.025442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:38.025745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.027154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.042205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.048122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 243/489] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:38.054453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.138876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.150458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32035b-32b0-4b59-823c-25fc381335c1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.190357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.191368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.191874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.192396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.192739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e282320-7eb1-406a-8c92-32814edc9c46 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.193498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:38.194049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.196520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2513a5a-9621-4a26-b467-aada1776a819 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.210956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 247/490] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:38.212830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a6a70-61cc-4e5e-ae17-ba2052972033 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.213342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a6a70-61cc-4e5e-ae17-ba2052972033 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.213868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a6a70-61cc-4e5e-ae17-ba2052972033 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.215882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 248/491] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:38.221094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a6a70-61cc-4e5e-ae17-ba2052972033 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.230721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.232554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.232840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.233111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.247316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.249576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-43f18d6d-fda8-467c-86f7-5f69133ef6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:38.265783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:38.266695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:38.267556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:38.267556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:38.267556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:38.267556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 244/492] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:38.268480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.268480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.268636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.286959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:38.288131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:38.331478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a6a70-61cc-4e5e-ae17-ba2052972033 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.339779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.372303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 245/493] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:38.372303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.378521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.379282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.379637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.379997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.395232 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.396837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:38.416043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:38.416973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:38.417880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:38.417880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:38.417880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:38.417880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 249/494] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:38.418952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.422773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.486233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ca6bf-0f5d-42f6-b89b-56bde13ffcf2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.534634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.535698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.536063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.536468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.538138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ff11-4bbf-4c61-9975-7963c1217443 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.551525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:38.556424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:38.557322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:38.558103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:38.562838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 246/495] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:38.563673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.579557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.580179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.580481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.580753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.581382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:38.581687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.583041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c77c09-b018-49a4-a1d4-de998a025489 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.600306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 250/496] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:38.600306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1518a-6e75-4279-806c-8a42a05b289e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.600306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1518a-6e75-4279-806c-8a42a05b289e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.600306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1518a-6e75-4279-806c-8a42a05b289e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.600306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 251/497] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:38.600703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1518a-6e75-4279-806c-8a42a05b289e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.620001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.677226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.690784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d59fa-4a11-4506-a365-a07ccec31c6f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.726439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1518a-6e75-4279-806c-8a42a05b289e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.735013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.735636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.735921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.736732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.737392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:38.737701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.739543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.752047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 247/498] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:38.757328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.763837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.764486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.764764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.765038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.765690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:38.765992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.767813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd7e7a-a579-47f3-b7c6-69ff1441aca8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.780374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 252/499] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:38.781660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.781956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.782240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.800657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:38.801852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:38.881800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 253/500] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:38.881800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.884248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27766130-6308-4740-a84e-1310f2690c3f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:38.886608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:38.928977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:38.929537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:38.929821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:38.930100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:38.942755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:38.944472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:38.959586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:38.960516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:38.961381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:38.964431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 248/501] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:38.965674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:38.990264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.038347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349760d5-8ddd-4a18-a9a6-76349dda778b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.073753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.074388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.074853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.074952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ab48-18ff-4d87-b35d-ba134c8479d7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.075357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.085700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.087814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:39.108164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:39.109099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:39.110050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:39.110050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:39.110050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:39.110050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 254/502] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:39.112310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.116172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.116856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.117134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.117408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.125313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.126768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:39.138801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:39.139669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:39.140573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 249/503] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:39.141139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.171592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.220323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707aa4-1f70-4453-89ff-e421389a992b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.229287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.246025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aca83-467f-45fd-bb1d-dada71f6e7d5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.255930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.256928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.257238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.257503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.269063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.270497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.280184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.280811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.281090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.281360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.282073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:39.282374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.283785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:39.284692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:39.285619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:39.286444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:39.286444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:39.286444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:39.286444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 255/504] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:39.286657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.320427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 250/505] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:39.321261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.394589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989e7b1-be6a-4553-a5cc-a6a04e061d30 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.405566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.424899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78309e-77b4-49da-a7d7-7343d79c2a52 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.427933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.428563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.428848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.429120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.429754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:39.430058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.431418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.443563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 256/506] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:39.444394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.462712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.463318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.463594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.463862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.464602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:39.464904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.465271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.466800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.484242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 251/507] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:39.484997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.547514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c6bbf-4c35-4355-9cec-08e472a00e55 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.581623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.582183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6022afe-bfe8-49be-ad32-3353580c8111 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.582296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.582526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.582801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.593537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:39.598416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:39.599296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:39.600104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 257/508] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:39.600789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.616398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.617000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.617280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.617551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.618179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:39.618477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.619847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.626695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.636065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 252/509] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:39.636065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.692950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.716251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d47d-3871-4c53-a09b-49a92f86ea88 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.732167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3896f-3c03-45c6-bfd1-5d8652988e80 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.750646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.751182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.751464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.751737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.771889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.772529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.772810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.773084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.773722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.775167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.796146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 253/510] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:45:39 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:45:39.797002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:39.840846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:39.843701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:39.869345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.923994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0041960-c152-4c01-9255-5d1472fd79b3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:39.940525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:39.961643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:39.962257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:39.962538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:39.962810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:39.963752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 258/511] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:39.965776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.047058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.048479 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.077436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba5ee-8bb3-4708-8c6b-88e54fc98fd2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.112120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.121600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.122214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.122493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.122766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.123422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:40.123729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.125127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.133318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 259/512] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:40.139157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.162157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 254/513] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:40.162999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.190569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.261872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a841617-3830-4219-8a7b-b83ff7798890 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.267432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d4bf4-a574-4159-a5d5-a62ab5b033d5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.300996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.301548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.301925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.302310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.303299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.303997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.304504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.304785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.310389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.311843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.316058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.317546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:40.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:40.325631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:40.326491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 260/514] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:40.327040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:40.337694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:40.338593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:40.339457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:40.339457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:40.339457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:40.342305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 255/515] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:40.347040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.354564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.442819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2625f3-e66c-4330-96ac-7f8faccedbef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.444355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.449056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b169b-0bfe-47ea-af23-44ced2b0e6bd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.484952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.485578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.485854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.486125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.486472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.487369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.488051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.488898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.496683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.498470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.502748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.504491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:40.513858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:40.514749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:40.515574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:40.515574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:40.515574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:40.515574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 256/516] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:40.515783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:40.527069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:40.528673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:40.529495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:40.529495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:40.529495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:40.534920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 261/517] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:40.534920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.591149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.636670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadcb64-5a40-47c8-876e-62666f2efc34 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.638032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b9ff7d-5881-41ad-983f-23641840c65e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.671980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.672637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.672923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.673201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.673492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.673836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:40.674092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.674187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.674376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.674649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.675369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:40.675519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.675677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.677274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.691946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 262/518] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:40.691946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.702197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 257/519] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:40.702197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.710923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.805422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93875-8a76-443d-a270-a86e802e5c9f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.805952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0cbe-c165-4b7b-9b57-4897699d3c29 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.824161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.843357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.844008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:40.844470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.844624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:40.844908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.845179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.845426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:40.845810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:40.846081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:40.846179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.846965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:40.847444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:40.847560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.849091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:40.858858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 258/520] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:40.859644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.865557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 263/521] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:40.866582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:40.945940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:40.966534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5e066-7992-479e-954a-32d7f2ee51dd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:40.977603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9b422-a178-43e7-a2a5-1c5f3538e605 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.002622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.003390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.003760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.004135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.011048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.013456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.016715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.017398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.017752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.018142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:41.025209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:41.026215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 259/522] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:41.032156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.064449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.114340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.116186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.139098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b646a8-963e-4714-a385-7306fc7dc2aa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.173766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.174398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.174682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.174958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.182510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.184532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:41.195879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:41.196988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 260/523] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:41.197859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.208523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.244151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 264/524] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:41.245080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.301058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.307022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfb15a-66e8-4e44-8018-e7b75c3c7bc7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.343657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.344225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.344546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.344828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.351923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.353606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.354442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11487bc4-f04b-499c-bd55-3de975ec1622 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:41.367344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:41.368386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:41.369404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:41.371038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 261/525] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:41.372406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.392992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.394026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.394540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.394751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.395448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:41.395756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.397227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.420896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 265/526] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:41.420896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.453471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.486048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7a0f2-3dea-451d-b707-3c21c9d29678 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.524858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.525474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.525756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.526029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.526755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:41.527063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.529169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.533202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29897299-66e6-41d0-9ee7-636cd3acc7d6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.553393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 262/527] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:41.553393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.567886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.575628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.576394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.576682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.576960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.587446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.588925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:41.604556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:41.605738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:41.607305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:41.607305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:41.607305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:41.607305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 266/528] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:41.607305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.665006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbd709-ab54-402e-b0f3-486d9134ee15 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.701362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.703202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.703852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.704147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.704452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.705366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:41.705475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.706864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.719895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d2514-6ad2-4858-9861-7a3d4f8c03af None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.730320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 263/529] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:41.730320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.756321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.756931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.757215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.757498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.758207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=03f7ea8f3fb64e3d96fbe277e07b5435) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:41.758517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.759939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.766705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 267/530] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/role_assignments?user.id=03f7ea8f3fb64e3d96fbe277e07b5435&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:41.767774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.819429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:41.851886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d72b56-63d0-4820-bb97-ba36c8f38e2a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.908952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.910298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.911575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.912398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.917709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e38c77-0016-4d65-91f1-cf1fb54d6b6a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:41.957684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:41.958301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:41.958589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:41.958866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:41.959507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:41.959817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:41.961247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:41.982300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 268/531] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:41.982300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:41.995043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.010915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.014116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.094874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.100702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeea507-15a0-468b-8fe1-bd641b89399f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.119416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 264/532] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:42.120515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.144895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.145919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.146373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.146896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.147778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=8d2458e0ed4b4fceb437ac5c7b43d598) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:42.148256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.150105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd2bfe-6794-415e-b191-042966a29b0b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.156250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 269/533] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:41 2022] GET /identity/v3/role_assignments?user.id=8d2458e0ed4b4fceb437ac5c7b43d598&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:42.157727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5d701-09a0-4b25-aa3f-be2d64682266 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.158139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5d701-09a0-4b25-aa3f-be2d64682266 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.158596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5d701-09a0-4b25-aa3f-be2d64682266 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.165003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 270/534] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:42.166272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5d701-09a0-4b25-aa3f-be2d64682266 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.255406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5852f-f063-4f64-8170-264d6b5f0c95 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.280328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.304587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.305290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.305574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.305848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.312090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5d701-09a0-4b25-aa3f-be2d64682266 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.315695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.316624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d7b1dcef-a803-4dae-837b-fa98c10fc920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:42.330047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:42.331132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:42.332175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:42.336441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 265/535] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:42.337842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.338615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.338899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.355857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.356561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.356815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.357089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.358855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:42.360062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:42.370979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.373211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:42.389439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:42.390364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:42.391192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:42.391192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:42.391192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:42.392813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.393219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 271/536] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:42.394323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.438531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 266/537] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:42.439798 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.523829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93035e7e-6115-40e2-b451-481b8a5f369b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.554583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.565517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.566406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.566833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.567339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.567578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9ea84-406a-487d-b728-380a29c3d143 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.568397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:42.568847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.570532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.584309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 272/538] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:42.591968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.640577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.643248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.643710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.644730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.646035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:42.647029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.649263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.676434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 267/539] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:42.682043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.728881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.792917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bf93d-2f75-4815-b1f1-04907457e185 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.833096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b9ca6-616d-45b6-9e2b-15961e2d68df None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:42.833927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.834536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.834817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.835091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.835920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=6e29455534ee474899b4f470d2a55f94) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:42.836231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.837787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.844565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 273/540] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/role_assignments?user.id=6e29455534ee474899b4f470d2a55f94&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:42.845369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.871574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:42.872760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:42.873273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:42.873764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:42.881226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:42.883044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:42.897283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:42.898178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:42.899006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:42.901499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 268/541] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:42.906586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.908816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:42.975503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e6e84-2d36-4e69-9d39-e878492a9bd8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.007914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:43.022545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.023588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.024423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.024824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.035663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3254df-33f3-4636-bcdb-f45a768bc719 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.046918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:43.052483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:43.053435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:43.054215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:43.054817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 274/542] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:43.061344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:43.137564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.138339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.138614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.138883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.139574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:43.139874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:43.141404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:43.175134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 269/543] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:43.180799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:43.215806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1063f0-ea1a-4925-9994-e19efc3132cc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.235054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:43.289300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.290283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.290799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.291284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.292141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:43.292678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:43.302001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:43.309140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 275/544] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:43.315042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:43.428144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d52f6-402e-4cc8-bb79-f52428587828 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.434475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e1bbc-9071-4e6e-b24a-aa62fcfbdab6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.468572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.469483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.469862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.470235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.482442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:43.484293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:43.500306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:43.501410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:43.502446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:43.502446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:43.502446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:43.504425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 276/545] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:43.509131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:43.516527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.517340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.517764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.518205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.519085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=53860f84e5cc4c84b15a8340f10c26d7) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:43.519538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:43.529475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:43.540346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 270/546] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:43 2022] GET /identity/v3/role_assignments?user.id=53860f84e5cc4c84b15a8340f10c26d7&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:43.540815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:43.634722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00a39c-e02d-42b7-9340-dacbdf05936a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.682051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.682727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.683008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.683432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.685076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:43.685578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:43.686988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:43.704404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 277/547] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:43.709857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:43.731843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:43.861373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d6f6-a6d7-4ac3-813b-758678d0080e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:43.914640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:43.915280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:43.915560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:43.915832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:43.926983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:43.928459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-94b549ed-d676-4ce2-9100-a5bb00439cc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:43.945825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:43.947239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:43.948714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 271/548] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:43.952971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:44.083115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:44.108431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88542408-08d4-4cd0-86cd-dc73cc2611b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:44.161771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:44.162716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:44.163219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:44.163712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:44.164675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:44.165184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:44.166796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:44.197973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130649a-d3e4-4e1c-a809-2e93e8f113cf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:44.198282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130649a-d3e4-4e1c-a809-2e93e8f113cf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:44.198531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130649a-d3e4-4e1c-a809-2e93e8f113cf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:44.201055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 272/549] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:45:44.206980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:44.207263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:44.207539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:44.217926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 278/550] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Sep 15 12:45:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:44.224627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:44.236025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:44.237293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:44.306538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:44.317087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 273/551] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:44.350204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a4408-f178-4819-b119-a06ed532ea93 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:44.398140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:44.398758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:44.399045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:44.399326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:44.416209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:44.495671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:44.498593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:44.626727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 279/552] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:44.632318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:44.675705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:44.727827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:44.807292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b99c-b24d-4d96-89ea-64b646ff18a1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:44.827036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6687691-05c3-4743-9fc1-02ab48dd3ba9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:44.872183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:44.872835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:44.873116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:44.873391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:44.888986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:44.889641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:44.889921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:44.890193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:44.895631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:44.897158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:44.917598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:44.922887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:44.923860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:44.925083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 280/553] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:44.930944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:44.936534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:44.937511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:44.938851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:44.938851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:44.938851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:44.940695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 274/554] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:44.946014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.026208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:45.077315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f6420-ad67-4d11-ad90-36a20e1345a2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.108149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3729a8-be29-4de8-bd61-9ed9a1496b73 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.110469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:45.129350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.129835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.130118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.130393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.131031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:45.131334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.132760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186ed9a-a29d-4c5d-b8a8-07896b271f9f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.147105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 275/555] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:45.154014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45fbd7-2215-4482-94c3-58154ca4d140 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.154344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45fbd7-2215-4482-94c3-58154ca4d140 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.154625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45fbd7-2215-4482-94c3-58154ca4d140 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.160054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 276/556] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:45.160054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45fbd7-2215-4482-94c3-58154ca4d140 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.160874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.161800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.162276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.162766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.163640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:45.164151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.165849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b321460-722b-4a32-9205-e603d502fcca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.177012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 281/557] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:45.178597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.179266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.179819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.202793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:45.203911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:45.274176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45fbd7-2215-4482-94c3-58154ca4d140 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.284359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 282/558] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:45.285566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.292605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:45.336915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.338095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.339602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.340933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.354095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.359178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:45.378209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:45.379133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:45.379981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:45.385051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 277/559] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:45.390979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.474290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:45.489083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8bffa-ebe8-4d2c-9e13-052d93c0e825 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.545826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.546473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.546744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.547014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.558029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.559959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:45.575943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:45.576999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:45.578443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:45.578443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:45.578443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:45.578443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 283/560] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:45.582597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.592874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4c200-4fd8-469e-a554-d3ed8e4cdd5b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.630502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.631112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.631395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.631670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.632390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:45.632923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.634147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe2b93-5677-4a8d-b8fe-97ad4eb28bdc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.654245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 278/561] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:45.654245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce3af2-617f-4b40-82d9-13a080d0127e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.654245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce3af2-617f-4b40-82d9-13a080d0127e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.654245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce3af2-617f-4b40-82d9-13a080d0127e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.656317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 279/562] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:45.656317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce3af2-617f-4b40-82d9-13a080d0127e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.688436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:45.703096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80163ae-9256-4162-a29b-b96d2f735a9a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.742255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.742955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.743247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.743525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.744172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:45.744763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.746549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.758520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 284/563] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:45.759345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.767088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce3af2-617f-4b40-82d9-13a080d0127e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.769444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:45.803191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db6b6-0dc5-40a6-8206-4eeb01cc04dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.803821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db6b6-0dc5-40a6-8206-4eeb01cc04dc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.804096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db6b6-0dc5-40a6-8206-4eeb01cc04dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.804400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db6b6-0dc5-40a6-8206-4eeb01cc04dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.816895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965db6b6-0dc5-40a6-8206-4eeb01cc04dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-965db6b6-0dc5-40a6-8206-4eeb01cc04dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:45.821745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:45.822644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:45.823484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 280/564] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:45.825145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.825145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.825145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.842620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:45.843824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:45.877602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6182fa-9a3b-4a47-ac88-409e34d96f79 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:45.915741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 281/565] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:45.915930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:45.916584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:45.916945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:45.917224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:45.917750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.917929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=4ead9001bd3241399dac6579a9588975) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:45.919118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:45.920611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:45.938556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 285/566] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/role_assignments?user.id=4ead9001bd3241399dac6579a9588975&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:45.939910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:45.960441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.086914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.138721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d2606-508a-4cb4-a1f8-50e7bc6a80f2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.144100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6154c-da61-4293-8043-8f4fb2b33486 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.189532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.190512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.191747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.192224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.194320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:46.194855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.196761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4fcfd-c8f8-4bcd-b10e-1051bcbbd04f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.202832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.203483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.203763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.204035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.209248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 286/567] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:46.210761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911101a9-cc80-47a7-909c-f34e4f5121cf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.211319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911101a9-cc80-47a7-909c-f34e4f5121cf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.211816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911101a9-cc80-47a7-909c-f34e4f5121cf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.213615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 287/568] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:46.214838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911101a9-cc80-47a7-909c-f34e4f5121cf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.218429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.219894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-058efe3a-cdc7-47ed-b414-57c992c971bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:46.237233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:46.238234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:46.239171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 282/569] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:45:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:46.240355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.240355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.240514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.259565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:46.260792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:46.336419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 283/570] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:46.337356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.342035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911101a9-cc80-47a7-909c-f34e4f5121cf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.351776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.382250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.389984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.390869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.391389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.391867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.404059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.405867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:46.421681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:46.422635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:46.423524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:46.423524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:46.423524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:46.425683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 288/571] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:46.426702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.469750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4bd52-1304-44e2-89c6-591273f6659e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.516745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.517420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.517709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.517982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.531999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:46.539299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:46.540243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:46.541123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:46.544423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 284/572] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:46.545555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.550295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac39de-d2d9-446b-a6b1-782b90360b8b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.587078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.588107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.588685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.589197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.590151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:46.590666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.592450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 289/573] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:46.615384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.642612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.663281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.681806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62404e4a-12e6-4ee9-990f-189e302c10d5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.722418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.723027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.723307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.723578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.724218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:46.724549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.725930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.727081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8105f4-968d-4da2-8fc5-da3e39b9fc48 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.737976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 285/574] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:46.737976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.770172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.770946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.771233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.771829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.772536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:46.772826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.774191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.787688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 290/575] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:46.787688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.855047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4319e4-c805-4428-b2fb-0bf19364fb54 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.886647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.893220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.893860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.894146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.894420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.897953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9640122-35d8-4177-ba48-492279c6ce88 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:46.901145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:46.905924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:46.907475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:46.922615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:46.923535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:46.929200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:46.929200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:46.929200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:46.929200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 286/576] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:46.929200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:46.942656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:46.943374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:46.943722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:46.944062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:46.955088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:46.960388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:46.961455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:46.962495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:46.968962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 291/577] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:46.969575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.036507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836247e-3d8b-45f4-a14b-586c230e9e66 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.069696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.070320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.070603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.070878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.078463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.079908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.082353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7256c-9d15-4750-84b1-732ef78d487d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:47.094599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:47.095694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:45:47.096888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:47.100486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 287/578] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:47.101382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.129187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.129839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.130124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.130398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.142144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.144007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.146944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:47.161276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:47.162208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:47.163090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:47.163090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:47.163090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:47.163090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 292/579] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:47.163983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.186682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.210446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf36b5f-0287-4360-94ae-1dfe09e78c48 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.245933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.246606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.246966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.247239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.247875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:47.248237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.249665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.267547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 288/580] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:47.267547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.281766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5f09-3f77-4899-a74c-e3a5f3f0037e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.319997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.320651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.320940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.321218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.321873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:47.322183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.324055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.337131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 293/581] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:47.337131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.375862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46cc5e-8d47-4b99-ac0b-743bdd8f4a82 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.377942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.414212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.414742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.415025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.415299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.416100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:47.416433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.417576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.417857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.436751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 289/582] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:47.437554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.446751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ff25d-4ad4-4e2f-8e24-ce67310d8799 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.482637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.483241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.483527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.483811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.484473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:47.484781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.486159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.498279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 294/583] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:47.499084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.550027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef86a22-ea22-4da3-9d91-eb0e364c1605 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.584765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.585398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.585681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.585960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.593306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.594763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:47.606652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:47.607562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 290/584] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:47.608871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.619485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5f600-4ccd-48c7-9dc6-208364924b83 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.631253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.659692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.660413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.660700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.660972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.669563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.674462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.675906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:47.693247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:47.694196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:47.695057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:47.695057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:47.695057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:47.695057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 295/585] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:47.696705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.715770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dea09e5-3369-42c5-af9a-1de8f872435a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.752534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.753607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.753895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.754333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.765263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.766729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:47.782613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:47.783605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:47.789202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:47.789202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:47.789202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:47.789202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 291/586] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:47.789202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.814911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1501635-8fe0-439a-bb9c-650a481cfe7a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.852588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.853440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.853903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.854178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.863104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.863675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.864926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:47.877698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:47.878677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 296/587] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:47.879610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.905768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47141-db96-429b-8c41-956da4dc1c0b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:47.934084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:47.940557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:47.941163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:47.941436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:47.941707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:47.942389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:47.942689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:47.944123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:47.957542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 292/588] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:47.958052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:47.994977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47df02-f7e2-462c-a9ad-71e4e9941948 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.034855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.035489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.035775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.036052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.036750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:48.037061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.038501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.052957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 297/589] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:48.055047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.071895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54febc-6157-442b-b2fe-d7a504f7f170 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.106215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.112326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.113013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.113388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.113759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.114498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:48.114893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.116409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.129313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 293/590] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:48.130288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.172081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89bb7b-d049-4ef0-b2db-dc754db9828a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.182529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.207658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.208260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.208568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.208841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.209551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:48.209855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.211272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.228882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 298/591] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:48.229670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.232218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876f4d4-d9ec-4843-b434-84c61a0c0f66 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.266524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.267194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.267471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.267745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/03f7ea8f3fb64e3d96fbe277e07b5435/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.281606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=03f7ea8f3fb64e3d96fbe277e07b5435, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.283055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.295292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 294/592] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:48 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/03f7ea8f3fb64e3d96fbe277e07b5435/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:48.296097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.344677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162c667-14b8-488d-9264-0b9ff613371a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.355534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.385959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.386972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.387339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.387825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.396602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.398063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b1a4502d-67e2-4f4a-b8de-8ad1860ae128 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:48.411348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:48.412416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:45:48.413358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:48.417401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 299/593] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:48.419414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cee0a-2a46-4b2c-9d51-ecf782f7c296 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.419871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cee0a-2a46-4b2c-9d51-ecf782f7c296 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.420153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cee0a-2a46-4b2c-9d51-ecf782f7c296 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.424321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 300/594] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:48.424321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cee0a-2a46-4b2c-9d51-ecf782f7c296 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4c537-e695-494d-9c80-02283a704e93 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.457203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.482620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.483281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.483631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.483977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.491344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.492879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:48.504740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:48.505645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:48.506516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 295/595] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:48.506991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.548395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cee0a-2a46-4b2c-9d51-ecf782f7c296 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.585567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.586611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.586882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.598661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.600230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.609695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb8e94-83db-47ef-8d56-39d252b7ce51 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.611361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-aa34d018-0b65-4d77-b1e2-e883a4abce70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:48.618012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:48.619155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 301/596] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.625433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.640635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:48.641818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:48.654724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.655537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.655819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.656088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.656847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:48.657147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.658587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.676201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 296/597] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:48.677051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.721228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 302/598] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:48.722487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.738751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.820707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e18e1-e0d3-4318-99af-633a0900f568 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.834988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53094e1c-ded6-4eab-9f11-2d4b6ee77843 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:48.869512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.870412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.870872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.871379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.871878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:48.872578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:48.872687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:48.872937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:48.873214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:48.873612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.873847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:48.874148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:48.875505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.875766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:48.888188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 303/599] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:48.888935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.892035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 297/600] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:48.897289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:48.930342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:48.965978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.001501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7391b-fcc8-46f0-93b3-22bf0a92b195 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.040613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.041246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.041526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.041794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.044243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c5905-51b2-43be-b20b-3425aba02edf None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.054496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.055937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:49.070724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:49.071669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:49.073023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:49.073023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:49.073023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:49.073023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 304/601] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:49.073023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.089165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.090649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.091092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.091397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/8d2458e0ed4b4fceb437ac5c7b43d598/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.111093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=8d2458e0ed4b4fceb437ac5c7b43d598, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.112572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.127486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 298/602] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:48 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/8d2458e0ed4b4fceb437ac5c7b43d598/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:49.128645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.211021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a548a-106f-4c3a-ae11-f45b406c0af4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.232910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.239422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.258842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.259833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.260315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.260790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.270196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.272066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.276416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a0774-a5d5-4160-a1b3-f362dcecfd45 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:49.286434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:49.287371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:49.288218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:49.290383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 305/603] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:49.291464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.320385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.321226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.321615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.321991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.337299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:49.342909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:49.344045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:49.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 299/604] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:49.345840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.426918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d578c6-5635-4cd4-8c29-fa89201d5061 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.466583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.467194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.467397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bf620-a676-47ab-b302-b299c9a28252 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.467512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.467733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.468395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:49.468697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.470170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.488087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 306/605] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:49.488087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.507328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.515648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.516199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.516510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.516783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.517472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:49.517826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.519262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.540308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 300/606] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:49.540308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.552438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.617609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75651270-d123-4344-a743-fdc580c97c2d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.655935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.657358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.657881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.658358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.659245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:49.659770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.661390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.671162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 307/607] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:49.671982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.679011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e186b-43b7-4796-a1f5-bcf6a0b53367 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.728422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.729474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.729750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.730016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.740817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.743419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:49.758784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:49.759735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:49.760772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:49.762285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 301/608] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:49.762891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.812938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc9ba-248d-4935-9283-3e90bbefaf7f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.816052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.816480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:49.854692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.855581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.856658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.857234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/6e29455534ee474899b4f470d2a55f94/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.871958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90046-b69e-4152-9f2e-9fce4bf6f0e6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:49.874511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=6e29455534ee474899b4f470d2a55f94, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.876026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.893431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 308/609] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:49 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/6e29455534ee474899b4f470d2a55f94/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:49.894437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:49.937822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:49.938811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:49.939335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:49.939785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:49.962442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:49.964302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:49.992267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:49.993307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:49.994187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:49.994187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:49.994187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:50.000660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 302/610] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:50.001946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:50.024351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bba35-fbd7-4425-ac4c-dc2fd255d476 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.062237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.063069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.063557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.063997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.064873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:50.065275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.066861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.081619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 309/611] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:50.088301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:50.110685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:50.160356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b87fa4-c8c5-42db-91af-3dee54121512 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.162165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:50.201272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.203535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.204033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.204534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.205379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:50.205899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.207502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.218554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b62873-c28d-4a4c-8e32-bcb3c781142e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.221490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 303/612] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:50.227098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:50.257546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.258671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.258971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.259245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/53860f84e5cc4c84b15a8340f10c26d7/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.275713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=53860f84e5cc4c84b15a8340f10c26d7, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.277362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.298395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 310/613] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:50 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/53860f84e5cc4c84b15a8340f10c26d7/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:50.372635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273799-3104-416e-b6f3-ae6bc7a10a84 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.419976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.420822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.421104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.421377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.429675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.431148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-fd9a096c-7c25-418c-b31b-5cbd63d964fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:50.445048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:50.446090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:50.447046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 304/614] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:50.454588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:50.539248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:50.580797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4387063-fe02-46d9-a9d5-72a6745b2ad0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.604033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257abdfc-2ba0-413d-a518-01cf8d872f3c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.604432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257abdfc-2ba0-413d-a518-01cf8d872f3c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.604612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257abdfc-2ba0-413d-a518-01cf8d872f3c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.606313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 305/615] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:50.611836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.612428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.612922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.623975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.624546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.624828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.625099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.625832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:50.626136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.627555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.637804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:50.640413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:50.648998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 311/616] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Sep 15 12:45:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:50.660223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:50.668707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:50.772356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 306/617] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:45:50.820192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23096663-37ad-4e96-914a-36de6b245edb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.844156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:50.856365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.857524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.857875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.858216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.868418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.869848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:50.885240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:50.886175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:50.887551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:50.887551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:50.887551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:50.887551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 312/618] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:50.893040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.042102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4a356-c5b8-4563-abb8-1d12b5431132 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.090196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.090815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.091095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.091356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.092008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:51.092334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:51.097760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:51.110482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 307/619] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:51.116552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.159833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.181001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:51.319145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f16a7-cf54-460b-8594-3f17bf47a5cb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.330048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2618f6-7e4b-4382-bdd5-dc1936471a46 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.371074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.371724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.372001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.372294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.389707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:51.396613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.397257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.397541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.397817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:51.400588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:51.401506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:51.403025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 308/620] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:51.413319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.414927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:51.416417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:51.437367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:51.439244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:51.441532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 313/621] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:51.446526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.523776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:51.528303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:51.584808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922adbcc-8415-4e5a-b844-f2e948d1fe28 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.651199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.651813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.652094 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.652461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.653134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:51.653438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:51.654827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c91c08-7acd-45b0-85e1-723cc6be44b1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:51.661221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ab882-4608-431d-969e-008ff7da8c80 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.671832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 309/622] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:51.673064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212705e-5d7e-4eb2-9b33-68b5a7864c34 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.673356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212705e-5d7e-4eb2-9b33-68b5a7864c34 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.673724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212705e-5d7e-4eb2-9b33-68b5a7864c34 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.675105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 310/623] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:51.679998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212705e-5d7e-4eb2-9b33-68b5a7864c34 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.702845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.703514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.703801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.704082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.704870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:51.705342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:51.706775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317ea3c-4c11-49bb-9aff-e02ff24bf852 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:51.724587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 314/624] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:51.724587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.724587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.724587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.741107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:51.742324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:51.807237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212705e-5d7e-4eb2-9b33-68b5a7864c34 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.827895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 315/625] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:51.828770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.845501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.846241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.846527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.846802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:51.849005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:51.858609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:51.862376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:51.865095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:51.881991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:51.882973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:51.883869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:51.883869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:51.883869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:51.885836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 311/626] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:51.894388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:51.943710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c68b8d-8729-452f-8a72-71109b2b1c4a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:51.983802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:51.984496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:51.984775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:51.985041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/4ead9001bd3241399dac6579a9588975/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.002000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=4ead9001bd3241399dac6579a9588975, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.004224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.025203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 316/627] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:45:51 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/4ead9001bd3241399dac6579a9588975/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:45:52.082821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648bb9a-3593-4f72-8e3e-501dbd2f7d2f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.141895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.142575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.142861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.143131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.143781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:52.144078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.145507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.160196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 312/628] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:52.167518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:52.208348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:52.268081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:52.296537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:52.303122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251034b-630b-410f-a9b6-5ad403ce00d5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.351490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.352131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.352467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.352736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.364721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.365331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c94d91-9e8f-43f9-bd53-eb862be000fd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.366923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:52.385107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:52.386177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:52.387444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:52.389973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 317/629] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:45:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:52.396000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:52.424435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.426413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.427444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.428835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.457202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:52.462067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:52.462997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:52.463842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 313/630] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:52.489346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:52.535177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971875c7-d95b-4a2b-91b0-7a510378f706 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.580308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.582077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.582810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.584024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.585235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:52.587982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.589787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.598337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:52.625472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 318/631] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:45:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:52.633883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:52.716368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917eacf-a861-4e71-8df7-705fa8acd4f4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.724627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:52.755108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.755845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.756192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.756569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.757445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:52.757838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.759399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8ff5f-f9fd-4b3c-8cb5-e9ab251988ce None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.769185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 314/632] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:52.770736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfa16a-bf02-4e22-ba29-d36f3f26ac3e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.771124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfa16a-bf02-4e22-ba29-d36f3f26ac3e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.771526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfa16a-bf02-4e22-ba29-d36f3f26ac3e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.773185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 315/633] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:52.778705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfa16a-bf02-4e22-ba29-d36f3f26ac3e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:52.779894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a52b2-f330-489d-bf8f-2ea5fb63163b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.819211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.819827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.820110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.820412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.890210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:52.895672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfa16a-bf02-4e22-ba29-d36f3f26ac3e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:52.901422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.903446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.934889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:52.935875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.936469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.936890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.948972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:52.951139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-87850c09-3fcc-4fea-947e-f27f43101ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:52.968022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:52.969239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:52.970275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:52.970275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:52.970275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:52.971691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 316/634] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:52.977629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:52.978121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:52.978600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:52.999051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:53.000589 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:53.019180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.027737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 319/635] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:53.028821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.079572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 317/636] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:53.080949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.131576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.149035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608a09b-07f6-4d66-b356-8fc543fbe4ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.200991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.201823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.202208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.202584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.203349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:53.203755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.205407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055dccd-bb06-48b5-9065-42fec16a27ac None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.217358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628a1b8-4636-4a0f-bee4-1de81ea1f904 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.219705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 320/637] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:53.224131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a872f-9478-4019-8387-15ddf127578a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.224131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a872f-9478-4019-8387-15ddf127578a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.224131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a872f-9478-4019-8387-15ddf127578a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.227122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 321/638] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:53.227986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a872f-9478-4019-8387-15ddf127578a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.268424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.269283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.269684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.270067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.281450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.283284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-fceb04d5-8dda-4031-b97e-7392de4cb5a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:53.298785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:53.299895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:53.300810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:53.300810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:53.300810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:53.300810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 318/639] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:53.302107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.302502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.302877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.309458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.323434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:53.325372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:53.356455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a872f-9478-4019-8387-15ddf127578a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.397616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.398627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.399136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.399633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.400581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 319/640] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:53.401523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.409219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.410927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:53.424048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:53.425046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:45:53.426014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:53.428837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 322/641] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:53.434496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.464628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.517540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e788-00be-4604-a3e1-2f9d132be05d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.547218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.554161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.554773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.555125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.555306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.555947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:53.556246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.557729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.560596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120b9d7-35be-4727-a78d-86c1e4cf4090 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.576059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 320/642] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:53.576989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.604012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.605117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.605797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.606284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.619347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:53.626007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:53.627025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:53.627881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:53.631955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 323/643] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:53.632903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.697987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88924ad4-f41a-487a-8234-97d45c3d4c32 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.710455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.733025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.733630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.733907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.734176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.734904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:53.735207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.736653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.746670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1575bd4-4fc2-4b50-bd70-eddc045b54d2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.758630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 321/644] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:53.758630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.787171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.788547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.788831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.789104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.796060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.797525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.807837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:53.811051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:53.812097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:53.813665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:53.816670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 324/645] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:53.818088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.866322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8a6b-887b-424b-9d6d-4238548a4610 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.902089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.902898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.903260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.903634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.904431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:53.904842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:53.906301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:53.919629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 322/646] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:53.919629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:53.933308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f3f09-7a38-48c2-84a8-c8e490548576 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:53.964175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:53.969865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:53.970613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:53.970995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:53.971367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:53.983015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:53.988395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:53.989494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:53.990484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:53.991576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 325/647] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:53.997540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.039310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7221bc-9f57-4165-96ad-9554a22cd664 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.053357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.076023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297956-d584-42de-a98a-43be607d152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.077044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297956-d584-42de-a98a-43be607d152d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.077178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297956-d584-42de-a98a-43be607d152d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.077520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297956-d584-42de-a98a-43be607d152d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.115309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92214c2d-20bc-427d-aad4-169d54e9775c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.151462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.152072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.152959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.153268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.154159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:54.154371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.155804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.156499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0297956-d584-42de-a98a-43be607d152d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.157940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0297956-d584-42de-a98a-43be607d152d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.170220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 326/648] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:54.170895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.228794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.276047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 323/649] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:54.276047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297956-d584-42de-a98a-43be607d152d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.285234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd167c-c844-44c7-ae23-54c088f47a4c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.288838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.325449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.326021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.326301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.326667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.338388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.341083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:54.358394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:54.359468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:54.360374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:54.360374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:54.360374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:54.365894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 327/650] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:54.365894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.385186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297956-d584-42de-a98a-43be607d152d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.422771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.423378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.423662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.423938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.424632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:54.424937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.426313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.435586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 324/651] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:54.436504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.471266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aecd5f-1da4-4434-80b0-501bb319df3b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.473584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.506129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.507488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.507787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.508094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.508866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=03f7ea8f3fb64e3d96fbe277e07b5435) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:54.509179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.510619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.512765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.517562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 328/652] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/role_assignments?user.id=03f7ea8f3fb64e3d96fbe277e07b5435&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:54.518364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.540200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1dfdf-d213-40b2-a495-a97a13eda78b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.579591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.580157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.582118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.582402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.594975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.596475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-abe92663-516d-49e5-b7da-04cfe2453ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:54.612088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:54.613182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:54.614063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:54.614063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:54.614063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:54.614063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 325/653] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:54.614741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb50e-4b4e-4732-991d-41eac5a88447 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.614928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb50e-4b4e-4732-991d-41eac5a88447 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.615128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb50e-4b4e-4732-991d-41eac5a88447 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.621388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 326/654] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:54.621388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb50e-4b4e-4732-991d-41eac5a88447 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.661843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ea3f-05f8-47c7-9a36-d9a8b04a3eb6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.702001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.702942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.703372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.703789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.704607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:54.705091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.706615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.721297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 329/655] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:54.722182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.740352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb50e-4b4e-4732-991d-41eac5a88447 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.759798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.774239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:54.784786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.785427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.785709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.785981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.796991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.798455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dbde578e-b819-4bac-9807-7c171540e7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:54.813341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:54.814348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:54.815237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:54.815237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:54.815237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:54.815237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 327/656] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:54.815956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.816204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.820653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.845068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452518c-f69e-4eef-b691-86c79a5bfe0b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:54.845574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:54.846705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:54.883673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:54.884372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:54.884629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:54.884907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:54.885547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:54.885853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:54.888206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:54.900312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 330/657] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:54.900774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:54.932450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 328/658] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:54.932450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.005259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.021442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e85b-6f43-49e7-a195-92ac81311416 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.045302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7c58-2709-4cfa-93d2-c571a9e422c7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.060336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.062653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.063299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.063589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.063861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.075837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.077320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:55.089848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:55.090884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:45:55.091795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:55.096196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 331/659] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:55.098198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.104497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.105432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.105921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.106420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.107305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:55.107871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.109609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.123631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 329/660] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:55.124790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.213175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8c019-fc67-4646-9da1-ec794a970b7c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.234612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261c1c2-bcf6-4582-84da-25d751e28efd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.250246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.250886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.251169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.251445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.258338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.259773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.270817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.271426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.271708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.271981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:55.272225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:55.273170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:55.277711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:55.278170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:55.278170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.278320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 332/661] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:55.278320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.278443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.288098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.297048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 330/662] 10.222.0.57 () {56 vars in 1137 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:55.297867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.328069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.389826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac9730-7741-47ab-9558-4fc1efb5db1e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.407130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052b469-a80d-4b3c-8617-ba4b7c9e788d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.425527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.426153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.426438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.426711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.435686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.437164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.443953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.444606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.444887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.445159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:55.448893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:55.450321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 333/663] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:55.451279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.460915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:55.466092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:55.467114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:45:55.467950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:55.472404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 331/664] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:55.473386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.531619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.562411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.570263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b192a-5098-4ed9-ae22-1ce651f1f570 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.578500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3759c-4f3d-41f7-abea-d98230736a90 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.614589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.614949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.615203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.615485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.615593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.615761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.615869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.616071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.616474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:55.616784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.618214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.640316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 334/665] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:55.640316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.687322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.688828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.753794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67b6f2-e700-4b1e-a944-622dcbd87d92 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.778616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.784309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:55.793687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.794305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.794587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.794860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.795587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:55.795892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.797377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.797983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 332/666] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:55.799064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.815469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 335/667] 10.222.0.57 () {56 vars in 1139 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:55.820555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.906840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfc520-7db4-4198-ba01-ebb14bfbfaf0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.922003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96066f80-1cfa-493d-8e43-faf2ee6a0eea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:55.942838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.943450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.943788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.944070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.944811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:55.945121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.946748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.960158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 333/668] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:55.960158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:55.962471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:55.963150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:55.963554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:55.963770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:55.964529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=8d2458e0ed4b4fceb437ac5c7b43d598) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:55.964870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:55.966290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:55.972121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 336/669] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/role_assignments?user.id=8d2458e0ed4b4fceb437ac5c7b43d598&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:55.972930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.026568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.041705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.095048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd82104-f467-482d-80ab-fa6dafd8d9ed None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.119979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db50b1-b10a-4e76-8854-b195168bb81b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.134219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.134868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.135432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.135551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.149290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.150794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.165077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.165639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.165899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.166167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:56.169404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:56.170317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:56.171227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:56.171227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:56.171227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:56.171227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 334/670] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:56.171459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.254560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.256196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.306202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.310811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e755b-8999-4217-8b0c-659079e5afb2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.314622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.360532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.361418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.361842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.362279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.374345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.376002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.379673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 337/671] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:56.380591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:56.393115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:56.394300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:56.395286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:56.395286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:56.395286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:56.402347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 335/672] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:56.403846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.520455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5ed8-5897-4f38-b073-dec1ac664bc8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.525236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870eb4-f322-488f-b153-fb23281e4867 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.557865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.559063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.559729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.560485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.570651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.571272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.571561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.572110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.572792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:56.573104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.573304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.574673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.575617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.593190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 338/673] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:56.593190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:56.598449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:56.599387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:56.600285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:56.600285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:56.600285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:56.602566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 336/674] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:56.603531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.603852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.609607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.723558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b58db5-89e3-413a-b4db-8c51423066b4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.732082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48f030-c35b-45eb-bf41-6f9aea8a16c3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.759925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.760608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.760883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.761160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.769039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.770531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.778678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.779607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.780105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.780610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.781468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:56.781991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:56.782845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:56.783702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:56.784639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 337/675] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:56.785118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.786525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9a1dd-3dc2-44ba-9841-af5e0afa47da None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.802834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 339/676] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:56.804226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fafbc6-2418-4a61-8237-5ec486d40bca None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.804641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fafbc6-2418-4a61-8237-5ec486d40bca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.805019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fafbc6-2418-4a61-8237-5ec486d40bca None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.806471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 340/677] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:56.808304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fafbc6-2418-4a61-8237-5ec486d40bca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.858192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.885024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:56.905857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9683f-c562-4c19-b973-407b430d09cc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.921372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fafbc6-2418-4a61-8237-5ec486d40bca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:56.940733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.941338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.941615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.941883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.942511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:56.942809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.944163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.959146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:56.959752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.960035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.960347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.961037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:56.961346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:56.962025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 338/678] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:56.962776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e5a2-3059-452f-82a6-864bb40e9723 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:56.962873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:56.980056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 341/679] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:56.980056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:56.980056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:56.980056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:56.995751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:56.997286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:57.072733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 342/680] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:57.073825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.080189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720048-cfba-44ba-935b-0b27e418f4b8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.116839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:57.128970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.129663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.130002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.130271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.138390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.139838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.148837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:57.151249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:57.152751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:45:57.153876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:57.157703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 339/681] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:57.158633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.204970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983b098-f5bb-4deb-b223-103457a301e7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.247321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.247951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.248227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.248525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.256485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.257672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:57.271114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:57.272013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:57.273018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:57.276688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 343/682] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:57.277635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.289155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b93d-3935-4bfb-9501-984cc7079772 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.334935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.335856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.336888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.337392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.338612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=6e29455534ee474899b4f470d2a55f94) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:57.339376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.341789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.356614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 340/683] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/role_assignments?user.id=6e29455534ee474899b4f470d2a55f94&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:57.358083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.396428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06161bc-6a73-4abc-8794-f505696b3328 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.420416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:57.438445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.439077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.439362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.439642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.440493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:57.440598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:57.440804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.442944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.466686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 344/684] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:57.467941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.516995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60edc71-d205-47a5-9e73-3c7beff299d4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.566846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.567461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.567747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.568022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.569180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:57.569496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.571306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.591146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 341/685] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:57.592144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.600055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253f3ea-c41e-4dcc-a2ac-454aaeddfea0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.647144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.647783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.648067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.648373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.662008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:45:57.667252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:57.668347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:45:57.669402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:57.671526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 345/686] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:57.672528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.724970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85d8e-94c8-404c-86d9-2c727e643dda None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.728858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:57.735515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:57.764814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.765694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.765975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.766924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.802654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82b81a-070b-4b34-85aa-6132b2ab82cf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:57.846035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:57.846569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:57.846851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:57.847125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:57.847851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=53860f84e5cc4c84b15a8340f10c26d7) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:57.848042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.848142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:57.849561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.849685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:57.856301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 346/687] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/role_assignments?user.id=53860f84e5cc4c84b15a8340f10c26d7&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:57.857155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.967385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 342/688] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:45:57.974026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:57.986291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.049058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534caa-a636-4bc1-92bf-a3acadd43254 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.052909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.124724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.125286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.125571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.125845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.126506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:58.126809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.128167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.141429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 347/689] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:58.149440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.157997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b81ad0-77d5-4ecb-acde-b3ff51088eee None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.197517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.198161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.198440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.200530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.210773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.212338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:58.228615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:58.229597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:58.230500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:58.230500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:58.230500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:58.230500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 343/690] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:58.235439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.273720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9491dc-24dd-47fb-93a7-dc6c19a99728 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.311931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.314914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.315581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.315831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.316113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.328594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.329527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:58.344310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:58.345306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:58.346284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:58.346284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:58.346284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:58.346284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 348/691] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:58.351205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.365988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c34d3-cc92-45ab-b429-896665e7b949 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.370626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.406735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.407737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.408344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.408843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.409730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:58.410244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.412029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074eebf-20f0-44f3-9f24-73dd6c343659 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.426256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 344/692] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:58.427805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e01d6-ea4f-4500-91e2-04ee46279861 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.428329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e01d6-ea4f-4500-91e2-04ee46279861 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.428807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e01d6-ea4f-4500-91e2-04ee46279861 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.430542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 345/693] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:45:58.437738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e01d6-ea4f-4500-91e2-04ee46279861 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.461204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b778b9-aff8-4ae0-9eea-851c63dcddd3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.498944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.499654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.499943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.500218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.500970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:58.501276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.502735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc0836-bdc1-4df2-94ff-e4d644538d72 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.516847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 349/694] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:45:58.522447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.522740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.523074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.546652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:58.548562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:58.565528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e01d6-ea4f-4500-91e2-04ee46279861 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.590108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.607401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.607961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.608247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.608996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.618179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.620345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:45:58.633679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:58.634735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:45:58.635711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:58.642754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 346/695] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:58.643898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.646699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 350/696] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:45:58.646699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.655213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.760733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c23079-b6fd-4e89-b6dd-6475aa9a6060 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.799990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.800690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.800973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.801300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.802621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:58.802936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.804402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.818856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 351/697] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:58.825410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.876777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4061a8-46d6-44af-8954-6f2cf6abd88f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:58.917049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:58.918611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:58.918900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:58.919173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:58.927475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:58.929008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:45:58.941477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:58.942453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:58.943952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 347/698] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:58.948166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:58.994964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:58.996366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:59.050552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ccb5-6744-42b0-9dcf-d3fcbba7e041 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.062278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18ed40-3e78-4141-ba47-6be650e1db40 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.102859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.103861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.104437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.104949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.106730 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=4ead9001bd3241399dac6579a9588975) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:59.107248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.107371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.107963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.108324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.108668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.109450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:59.109815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.109996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.111340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.123452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 352/699] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/role_assignments?user.id=4ead9001bd3241399dac6579a9588975&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:59.123452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:59.147981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 348/700] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:45:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:59.152496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:59.330521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63349f-9b73-4bf6-8c20-0ba7fc050067 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.346392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:59.351521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a542af-1674-4ecd-9d0a-d5835bb6e7ac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.375694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:59.385532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.386867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.387426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.387870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.388882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.391220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.394466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.395172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.395546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.395909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:59.406818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:59.407884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:45:59.408862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:45:59.411387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 353/701] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:45:59.419114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:59.482609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.484463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.554948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e3e6a-5aa3-42c5-9f7f-75d996c8909f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.603760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.604830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.606183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.607034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.608163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:59.608690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.609055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:59.610971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.616146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 349/702] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:45:59.617419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 354/703] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:59.623025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:59.625252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:59.692738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:59.731711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec5f7b-8d3c-4740-a6b8-02de02935875 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.744677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9120-52a5-48bf-9537-1e3b85add323 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.770719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.771382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.771666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.772014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.785026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.786523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.795248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.796320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.796837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.797304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.798135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:59.798649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.800330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:45:59.809197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:45:59.812826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:45:59.814009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:45:59.814009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:45:59.814009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:45:59.814009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 350/705] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:45:59.814429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 355/705] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:59.815044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:59.874377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:59.935447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a27d8-e28e-4456-8517-07e5b8660aae None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:59.973995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:59.974614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:59.974895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:59.975171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:59.975878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:45:59.976185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:59.977690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7a462-b89a-43aa-beab-47e458314b1a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:59.992306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 351/706] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:45:59.999116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:00.057423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b609f4-7bbb-4638-9e47-f68a34978078 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.057923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b609f4-7bbb-4638-9e47-f68a34978078 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.057998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b609f4-7bbb-4638-9e47-f68a34978078 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.059949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 352/707] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:00.067682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.067976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.068256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.102271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:00.103718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:00.132695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71ff1-87d7-48ca-ae82-2779b327cc32 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:00.161886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:00.199472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 353/708] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:46:00.205000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:00.205669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.205955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.206234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.217222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:00.218674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:00.241784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:00.242868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:00.244019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 356/709] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:45:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:00.249116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:00.314307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:00.389805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63afc6-520a-4579-a09a-43e08029551f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:00.442791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:00.443425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.443710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.443986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.444724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:00.445032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:00.446491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:00.463580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 354/710] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:00.469291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:00.537821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:00.594040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d149-9c94-41b5-bd2f-e8613f1929e3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:00.631776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:00.632416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.632701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.632976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.633697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=03f7ea8f3fb64e3d96fbe277e07b5435) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:00.634007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:00.635431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f9bc4-0809-4a35-82ec-3e1bb4675aa9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:00.643481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 357/711] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:00 2022] GET /identity/v3/role_assignments?user.id=03f7ea8f3fb64e3d96fbe277e07b5435&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:00.652670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059c8d-fbd5-440d-b640-a3a001548ff6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.653656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059c8d-fbd5-440d-b640-a3a001548ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.654158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059c8d-fbd5-440d-b640-a3a001548ff6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.656343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 358/712] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:00.662408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.663298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.663758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.679706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:00.687238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5ad23-c4b4-4d23-918d-70fcd9485eed None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:00.694848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:00.696407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:00.756666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:00.780473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 359/713] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:46:00.793498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:00.794093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:00.794376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:00.794650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:00.820898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:00.823148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:00.850906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:00.852019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:00.852944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:00.852944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:00.852944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:00.852944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 355/714] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:00.858681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:00.990809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ec090-f704-40bc-88fa-535b8382e4e2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:01.029768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:01.030728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:01.031235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:01.031705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:01.032563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:01.033099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:01.034752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:01.047311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 360/715] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:01.057502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:01.103203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:01.142920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:01.215631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ae4d2-00c5-41dd-8670-19f09f79124e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:01.251695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:01.252398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:01.252682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:01.252956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:01.253689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:01.255060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:01.265881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:01.266885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:01.268112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 361/716] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:01 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:01.272991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:01.318343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498a5de-e7a7-4ec2-8589-4f7b957d558a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:01.380299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4f98f-77e2-43cd-aec7-5d2d855986ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:01.388645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:01.419437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:01.420057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:01.420384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:01.420666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:01.421295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] RBAC: Adding query filter params (name=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:01.421606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:01.423051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:01.427793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 362/717] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:01.434347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:01.452989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aa8e4-e172-406b-8b96-5b6b2655cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:01.453660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aa8e4-e172-406b-8b96-5b6b2655cc61 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:01.453935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aa8e4-e172-406b-8b96-5b6b2655cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:01.454343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aa8e4-e172-406b-8b96-5b6b2655cc61 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:01.456186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aa8e4-e172-406b-8b96-5b6b2655cc61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:01.457659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aa8e4-e172-406b-8b96-5b6b2655cc61 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:01.500394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 356/718] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:46:01 2022] POST /identity/v3/users => generated 254 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:01.590221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95734aa6-d11e-4c2f-9739-887a0fd2a748 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:01.649072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:01.649715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:01.650067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:01.650346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:01.651034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:01.651345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:01.653109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8c41e-da54-4248-aabb-ef33b9347c39 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:01.658897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 363/719] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:02.224082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea081b38-653b-42b6-86f4-b5f7da0909c0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:02.224637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea081b38-653b-42b6-86f4-b5f7da0909c0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:02.224713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea081b38-653b-42b6-86f4-b5f7da0909c0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:02.226372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 357/720] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:02.234406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3879-2a6f-4f45-8b5f-755bac1b8538 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:02.234974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3879-2a6f-4f45-8b5f-755bac1b8538 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:02.234974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3879-2a6f-4f45-8b5f-755bac1b8538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:02.264978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9a0a3879-2a6f-4f45-8b5f-755bac1b8538 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:02.266245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9a0a3879-2a6f-4f45-8b5f-755bac1b8538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:02.294327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:02.352409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 364/721] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:46:02.353099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02608e6a-b8fc-4159-a715-ea724a45d992 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:02.353388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02608e6a-b8fc-4159-a715-ea724a45d992 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:02.353661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02608e6a-b8fc-4159-a715-ea724a45d992 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:02.355007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 358/722] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:02.360118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:02.360434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:02.360715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:02.383657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:02.384998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:02.455663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 365/723] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:46:02.702666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02608e6a-b8fc-4159-a715-ea724a45d992 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:02.715835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:02.832147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02608e6a-b8fc-4159-a715-ea724a45d992 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:02.871003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:02.872185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:02.872674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:02.873093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:02.885651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:02.892366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:02.893478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:02.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:02.896832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 359/724] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:02.904301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:02.947393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:03.023716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:03.064816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a77c20-1fe2-4e34-954a-ffbe43e6d48c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:03.103410 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:03.104026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.104333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.104613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.105804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:03.107192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:03.128660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 360/725] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:02 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:03.146638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:03.174802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6f76-99b3-4282-938f-41b95ff741ab None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:03.249517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:03.250299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.250457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.250663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.251330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:03.251635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:03.253036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:03.266158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 366/726] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:03.272422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:03.418708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd129b8-b1be-4fdb-a747-8351b9cc6ee0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:03.450127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:03.459473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:03.460384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.460852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.461357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.479815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:03.481395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-fea00ed4-7bef-4ce1-a178-b34c9caa2608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:03.500804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:03.503376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:03.505517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:03.505517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:03.505517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:03.505517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 361/727] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:03.508093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:03.623507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712f2d0-1e4a-4180-aafe-1499e49af331 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:03.635659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2dad6-34cd-4ae9-9a05-861582f0369b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.635942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2dad6-34cd-4ae9-9a05-861582f0369b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.636218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2dad6-34cd-4ae9-9a05-861582f0369b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.638295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 362/728] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:03.643466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.643749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.644020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.669037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:03.670454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:03.673996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:03.674869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.675347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.675829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.676678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:03.677161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:03.678684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:03.693103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 367/729] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:03.699496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:03.710667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:03.756200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 363/730] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:46:03.819157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab62541-ae5f-40ea-bc52-d6a230e83ed7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:03.864507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:03.865472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:03.865931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:03.866401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:03.875159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:03.876885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:03.890112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:03.891088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:46:03.892017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:03.894207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 368/731] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:03.901356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.052572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5db23c-a223-48f0-965b-f41f68959e83 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.065031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.085727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.091082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.091704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.091984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.092262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.093019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:04.093326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:04.094808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:04.114178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 364/732] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Sep 15 12:46:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:04.121544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.232632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.256875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e1f0e-2ea3-468e-8d7c-0ef0551aeef4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.273219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38612a7d-cba6-4197-b2b4-430d962eb34f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.295676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e449-a04b-4727-9f0f-425380269c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.296718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e449-a04b-4727-9f0f-425380269c48 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.297091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e449-a04b-4727-9f0f-425380269c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.297702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e449-a04b-4727-9f0f-425380269c48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.310028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8363384-2f50-4587-b860-2f4eb943fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.310603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8363384-2f50-4587-b860-2f4eb943fee0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.310884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8363384-2f50-4587-b860-2f4eb943fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.311157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8363384-2f50-4587-b860-2f4eb943fee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.322459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8363384-2f50-4587-b860-2f4eb943fee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a8363384-2f50-4587-b860-2f4eb943fee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:04.329535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:04.330681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:04.331551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 369/733] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:04.332227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1782b2-7550-466d-a812-0d67c3355673 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.337069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1782b2-7550-466d-a812-0d67c3355673 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.337378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1782b2-7550-466d-a812-0d67c3355673 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.338881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 370/734] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:04.339642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1782b2-7550-466d-a812-0d67c3355673 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.347320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.389756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e449-a04b-4727-9f0f-425380269c48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:04.391743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e449-a04b-4727-9f0f-425380269c48 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:04.461487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1782b2-7550-466d-a812-0d67c3355673 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.488435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.499022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.499645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.499924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.500194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.500873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:04.501175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:04.502532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:04.508395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 365/735] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:46:04.509952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.510543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.511013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.517668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 371/736] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:04.518566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.531746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:04.532978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:04.596666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.602282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 366/737] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:46:04.603098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.653713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ca486-9f0d-4fa2-b5c9-1b787e98a956 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.700893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.701539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.701821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.702091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.712773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:04.714221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-db4ecefe-0f25-4093-87a3-69e755b0bd48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:04.729869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:04.731389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:04.732258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:04.732258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:04.732258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:04.732258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 372/738] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:04.734031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57620-81fd-46e8-803e-3eaeeb5ce6a4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.734031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57620-81fd-46e8-803e-3eaeeb5ce6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.734031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57620-81fd-46e8-803e-3eaeeb5ce6a4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.735440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 373/739] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:04.740219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57620-81fd-46e8-803e-3eaeeb5ce6a4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.742643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0129e60-1bc0-450f-8043-b91da0a377a1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.784059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.784747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.785072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.785348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.796687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.797827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:04.799566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dd70b13e-6bf0-4216-a985-2ce995bbcb33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:04.817268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:04.818237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:04.819129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:04.819129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:04.819129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:04.823272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 367/740] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:04.824917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.825277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.825625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.846138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:04.848669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:04.867488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57620-81fd-46e8-803e-3eaeeb5ce6a4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:04.904481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:04.908199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:04.908890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:04.909180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:04.909459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:04.910101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:04.910408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:04.911829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:04.924570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 374/741] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:04.925220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:04.936302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 368/742] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:46:04.936302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.032819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.036384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649048cc-6e38-4c76-b6de-3e9f373486fd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.057176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f5b1-890b-43ae-9763-0c12635883d5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.072883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.073505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.073874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.074312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.075181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:05.075794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.077199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.090707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 375/743] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:05.091903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.107170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.108032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.108499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.108995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.109879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.111482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.128740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 369/744] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:04 2022] POST /identity/v3/services => generated 240 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:05.129863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.179629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.218926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b1591-2c12-4a88-9c75-ded2383eeff2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.259262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a621-5ddb-4376-974c-b7d0f78c3182 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.262118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.262927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.263243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.264250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.271369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.272945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:05.286660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:05.288362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:05.289437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 376/745] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:05.294057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.304758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.306136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.306576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.307008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.316445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.317029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.319411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:05.334645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:05.335858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:05.336901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:05.339198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 370/746] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:05.341258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.409385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309456-b4fe-4a59-9913-1cfd7825b88b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.451240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.451591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.451883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.452354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.452471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.459239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48919-2a7e-40d9-ae31-4564e8799ee8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.463476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:05.469549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:05.470716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 377/747] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:05.471817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.504787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.505548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.505930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.506301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.507093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:05.507498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.509218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.523305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 371/748] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:05.524152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.557800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.583215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d1739-21e1-4bdf-ab42-d34bda8ebd25 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.619640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.620309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.621299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.621543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.622281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:05.622590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.624022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.649089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 378/749] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:05.649089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.650126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3574066-dfae-4669-9e6d-b7e5bba36ae2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.688709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.689858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.690574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.691153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.692198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:05.692738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.694725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.710108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 372/750] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:05.711432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.733530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.754740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5a76a-9b6d-4b35-8f05-53e629276a78 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.792013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.792711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.792993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.793267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.793992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=4ead9001bd3241399dac6579a9588975) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:05.794302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.795821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.797771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:05.802392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 379/751] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/role_assignments?user.id=4ead9001bd3241399dac6579a9588975&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:05.809175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:05.830123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df45f3-77f4-4a16-863f-bbb7208e4b3f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:05.888306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:05.889411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:05.890195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:05.892934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:05.919773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:05.924947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:05.942583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:05.943559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:05.944482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:05.944482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:05.944482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:05.946541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 373/752] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:05.953703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.038466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d493822-317d-42d2-aee7-b7778d2cf786 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.067426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:06.084477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.085502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.085904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.086583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.102113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.103581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.103766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a09e9-3205-43db-86d6-86a3a7a7844b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1b6f424c-ef86-408b-8fdd-c0da028adc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:06.124210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:06.125200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:06.126117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:06.126117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:06.126117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:06.126285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 380/753] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:06.128995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf857c7-8e55-4d25-b658-f1d0d5a04ef2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.129389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf857c7-8e55-4d25-b658-f1d0d5a04ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.129779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf857c7-8e55-4d25-b658-f1d0d5a04ef2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.131427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 381/754] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:06.132500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf857c7-8e55-4d25-b658-f1d0d5a04ef2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.151753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.153010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.153297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.153853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.154512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:06.154820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.156886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3280c-102c-4ea5-8b7b-ff85b2a2b408 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.175643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 374/755] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:06.177008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.177301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.177574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.196183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:06.205276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:06.206506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:06.273728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf857c7-8e55-4d25-b658-f1d0d5a04ef2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.279147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 375/756] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:06.280105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.320212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.321956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.322446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.322945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.324980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:06.325968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.327837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.341829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:06.348435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 382/757] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:06.351080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.389536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129aeed-94d7-43e3-b349-0b71562ce55d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.426344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.426981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.427261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.427532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.438384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.439901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:06.452694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:06.453664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:06.454530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 376/758] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:06.459703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.488769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8437c-2960-441a-909c-6c1547a4c4d7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.502588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:06.532425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.533606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.534109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.534583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.542707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.544528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:06.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:06.557936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:06.559391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:06.562835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 383/759] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:06.570288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.669333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129e711-9b4d-456c-a824-830b81029b99 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.697200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:06.718435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.719898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.720651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.721129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.722627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:06.723401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.725695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.728095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46095d0e-4f92-421e-a017-b3442200d8cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.742870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 377/760] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:06.744529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.766750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.767392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.767671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.767941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.768783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:06.769065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.770487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.792473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 384/761] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:06.792473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.823090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:06.887738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730401-bf5a-46a0-a21c-2e2aafed1d3b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.923343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fa4eb-f846-4b3d-88e6-cf1cca0303be None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:06.934911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.935682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.936038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.936411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.937061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.938510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:06.950335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:06.951318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:46:06.952133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:06.956363 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 378/762] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:06.957258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:06.966700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:06.967902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:06.968189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:06.968551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/03f7ea8f3fb64e3d96fbe277e07b5435/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:06.971314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=03f7ea8f3fb64e3d96fbe277e07b5435, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:06.973064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:06.993505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 385/763] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:46:06 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/03f7ea8f3fb64e3d96fbe277e07b5435/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:46:06.994678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.023551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:07.114933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:07.133801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8d8f-e223-44c0-800d-e37afb1be0f6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.176603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.177228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.177513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.177788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.178428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:07.178731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.180113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.182597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6b36c-5ee8-4cbe-898a-a0797ef36840 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.186771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 386/764] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:07.193429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.264220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.264793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.265071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.265342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.349938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.351417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.352496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288291a2-9f26-4f88-a050-5deacde56397 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.398376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.399288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.399575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.399852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.400624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:07.401202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:07.401601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.403089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.409914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 387/765] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Sep 15 12:46:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:07.415712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.450178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:07.466063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 379/766] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:07.471637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.584668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01445-4ab3-4502-90c6-d861e5ad377f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.595214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148bd76-0972-4c37-8766-a337d6e4d383 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.623631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.624397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.624671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.624941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.636988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.638974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.639204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.639614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.639903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.640182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.640972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] RBAC: Adding query filter params (interface=public, service_id=4e3e8bfad6f74d84be9aaa9e8ef920a0, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:07.641288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.641666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:07.642650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.660066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 388/767] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:07 2022] GET /identity/v3/endpoints?service_id=4e3e8bfad6f74d84be9aaa9e8ef920a0&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:07.663431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:07.665673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:07.667989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:07.667989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:07.667989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:07.667989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 380/768] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:07.669513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.672328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.775611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:07.784537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35e6d-b059-4e55-9796-302f6cdf391d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.800297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ad858-0af0-4cfe-8675-835ac0c81349 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:07.828413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.829050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.829333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.829605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.830244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:07.830545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.831915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba252483-4a98-4f4e-b144-0f744998f7dd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.844616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:07.845558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.846221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.846330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 381/769] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:07.846798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.847585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365c61-edf0-40b3-9faa-02de94bef602 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.847824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:07.847936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365c61-edf0-40b3-9faa-02de94bef602 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.848132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81365c61-edf0-40b3-9faa-02de94bef602 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.849742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:07.853635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 382/770] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:07.855785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59496970-9787-48ce-a260-a3c0784565de None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:07.856134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59496970-9787-48ce-a260-a3c0784565de None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:07.856559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59496970-9787-48ce-a260-a3c0784565de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:07.857571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 389/771] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:07 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:07.864351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:07.892994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-59496970-9787-48ce-a260-a3c0784565de None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:07.896789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-59496970-9787-48ce-a260-a3c0784565de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:08.016701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:08.044812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61254b59-aa0b-4046-96eb-d2e96e17ae94 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:08.065949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 383/772] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:08.100353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:08.101243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.101720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.102240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.110643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:08.112504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-78d8250f-c773-4c53-9fb2-030df82abfba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:08.124845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:08.125751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:08.126694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:08.129134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 390/773] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:08.135090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59496970-9787-48ce-a260-a3c0784565de None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:08.251157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59496970-9787-48ce-a260-a3c0784565de None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:08.287422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:08.288440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.288918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.289417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.290311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:08.290820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:08.293009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:08.298979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e2e13-a772-4bd1-91a7-f414e4318204 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.299551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e2e13-a772-4bd1-91a7-f414e4318204 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.300025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e2e13-a772-4bd1-91a7-f414e4318204 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.301555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 391/774] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:08.306475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.306762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.307039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.312110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 384/775] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:08.324350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:08.332631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:08.335423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:08.336713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:08.406035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:08.440549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 392/776] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:08.441526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:08.472090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb3baa-51a3-4511-9595-3971bcd4b672 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:08.509896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:08.510667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.511054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.511503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.512863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=61362637fe434cbeba9d2d9efe37605a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:08.513278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:08.514994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:08.521388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 385/777] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:08 2022] GET /identity/v3/role_assignments?user.id=61362637fe434cbeba9d2d9efe37605a&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:08.654838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bf8fa-b5a6-410c-9d83-730184806440 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:08.709551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:08.710213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.710493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.710766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.711359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:08.712785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:08.714953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:08.723813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:08.724893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:08.725761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:08.728530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 393/778] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:08 2022] GET /identity/v3/services/network => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:08.734930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:08.795253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:08.921991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5058899-57a0-4990-b7f6-8510d6838ff6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:08.960845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:08.961446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:08.961777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:08.962075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:08.973824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:08.979207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:08.980254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:08.981248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:08.984404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 394/779] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:08.986977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.018252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5678db6-15c2-496e-bd56-a75d4820f97e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.050225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:09.070394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:09.114568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d4f21-7c4d-43a1-9bb9-2c1d5334c359 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.121041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.121791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.122176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.122548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.123283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:09.124005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.125290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.134040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 386/780] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:09.141014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.156577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.157442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.157922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.158346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.159198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:09.159628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.161218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.171700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 395/781] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:09.177145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.315509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ce507-0ee4-4998-bf2a-3284a4ea6d58 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.342834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:09.372458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b66d74-4c09-49f0-8bef-99524330da6f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.372987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.373641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.374093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.374373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.395481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.396966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:09.423851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:09.424884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:09.425761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:09.425761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:09.425761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:09.432230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 396/782] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:09.434006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.434576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.434840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.435126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.435746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] RBAC: Adding query filter params (type=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:09.436043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.437471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.439266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.456346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 387/783] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:09.462097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.522758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:09.639540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ce19-8c95-4293-a323-0cf7cd980cec None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.654331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda571a3-1863-4043-a974-5b13d0893ef1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.696641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.697644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.698423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.699436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.701050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] RBAC: Adding query filter params (interface=public, service_id=3bdcf15969404cf8adec0f66917b54d8, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:09.702084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.705825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.721965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 388/784] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:09 2022] GET /identity/v3/endpoints?service_id=3bdcf15969404cf8adec0f66917b54d8&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:09.729781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.748148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.748795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.749080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.749355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.749997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:09.750306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.752052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.775525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 397/785] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:09.779254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:09.826432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:09.883587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a8d46-534b-4499-bbd1-70b589ba0cd2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.885295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:09.903009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56dae11-6bde-47fb-be5b-e18fbafccb08 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:09.931538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8351ee-9439-4e9f-b2dc-18fb811f5360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.932208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8351ee-9439-4e9f-b2dc-18fb811f5360 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.932497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8351ee-9439-4e9f-b2dc-18fb811f5360 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.932788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8351ee-9439-4e9f-b2dc-18fb811f5360 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.933982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8351ee-9439-4e9f-b2dc-18fb811f5360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.935505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8351ee-9439-4e9f-b2dc-18fb811f5360 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:09.945753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 389/786] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:09 2022] GET /identity/v3/services => generated 772 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:09.948070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe425ef-e61f-4cf0-9dac-d3eb6ea70ed6 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.949303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe425ef-e61f-4cf0-9dac-d3eb6ea70ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.951183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe425ef-e61f-4cf0-9dac-d3eb6ea70ed6 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.952923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 390/787] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:09.953909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:09.954815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.955343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.955823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.963353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:09.963997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:09.965195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:09.973661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:09.982700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:10.009920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:10.010935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:46:10.012507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:10.015275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 398/788] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:10.020761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:10.020761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:10.026461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:10.154197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:10.167015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7f4d-2630-4d75-bb20-39167aab38b7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:10.194039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 391/789] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:10.219326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:10.220186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.220643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.221100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.221978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:10.222432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:10.224026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:10.246585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 399/790] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:10.251989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:10.353057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:10.364259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a6a90-35a0-49d3-8695-927264241b48 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:10.421470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:10.422457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.422957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.423447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.444103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:10.446397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:10.472433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:10.473431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:10.474329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:10.474329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:10.474329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:10.480748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 392/791] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:10.485435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:10.537354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:10.645846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4660fb-d15e-4d1e-a6e5-e4023edacfd3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:10.672153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9798f0f-20c8-4304-8b37-3763558ed07c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:10.688308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:10.688901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.691478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.691478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.691478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:10.691478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:10.692069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09b73c-c088-428b-9771-cb7b54963784 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:10.708348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 400/792] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:10.709680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2838339-a1dd-476c-a225-3d7c6ee456c9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.709979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2838339-a1dd-476c-a225-3d7c6ee456c9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.710256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2838339-a1dd-476c-a225-3d7c6ee456c9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.711782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 401/793] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:10.717055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2838339-a1dd-476c-a225-3d7c6ee456c9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:10.721045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b0ec2-8ddc-4de4-9781-31cfddd5f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:10.721958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0ec2-8ddc-4de4-9781-31cfddd5f293 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.722439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0ec2-8ddc-4de4-9781-31cfddd5f293 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.722925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b0ec2-8ddc-4de4-9781-31cfddd5f293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.730837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:10.737095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b0ec2-8ddc-4de4-9781-31cfddd5f293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ec7b0ec2-8ddc-4de4-9781-31cfddd5f293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:10.744058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:10.745155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:10.746260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:10.749156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 393/794] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:10.750556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.751055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.751526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.774572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:10.776416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:10.781250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:10.847848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2838339-a1dd-476c-a225-3d7c6ee456c9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:10.888079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 394/795] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:10.888079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:10.897093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:10.897736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.898019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.898342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.916432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:10.919592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-fa344ef0-14c9-46a1-adda-23918de5ce41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:10.941480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:10.942400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:10.943291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:10.946673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 402/796] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:10.948010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b186-af85-4007-b5d1-9875afb89471 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:10.948314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b186-af85-4007-b5d1-9875afb89471 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:10.948933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b186-af85-4007-b5d1-9875afb89471 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:10.950856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 403/797] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:10.951562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b186-af85-4007-b5d1-9875afb89471 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.029870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9938-2c5f-4835-9660-6d57716ee7ed None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.037671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.072628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.073237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.073551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.074031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.074709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:11.074827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b186-af85-4007-b5d1-9875afb89471 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.075021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.076562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c4b76-3801-435b-9a9c-e5b112d82299 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.086375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.093376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 395/798] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:11.094925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.095391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.095808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.121389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:11.122835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:11.125693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.127648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.128093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.128539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.129401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:11.129882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.131503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.149304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 404/799] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:11.150374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.208779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 396/800] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:11.208779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.270288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f687f-993e-4145-b3f1-e0352468f5fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.301332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.315441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.316455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.316885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.317431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.330244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.332373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.338044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b72524-8751-4508-82ca-0d4853bdd13e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:11.350736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:11.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:11.352752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:11.352752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:11.352752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:11.352752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 405/801] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:11.352752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.372334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.379687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.380690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.381156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.381607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/4ead9001bd3241399dac6579a9588975/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.383837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=4ead9001bd3241399dac6579a9588975, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.385680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.401564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 397/802] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:46:11 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/4ead9001bd3241399dac6579a9588975/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:46:11.403740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.529768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9fd92d-7c24-4361-bb82-e2d9f0a8edd6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.570383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.571286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.571644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.572131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.572864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.575009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.580041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09702d05-2069-4c91-aad9-ddbd2f5cd8c2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:11.585146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:11.586157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:11.586983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 406/803] 10.222.0.57 () {56 vars in 1129 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:11.587777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.627938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.628557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.628843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.629168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.629841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:11.630150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.631607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.645494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 398/804] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:11.646401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.651875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.719938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d314c2-be17-4628-a21b-8dafd33adde9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.720995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.767311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a226e-02d2-4f8e-bf1c-123d82f9ccb5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.767893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.768568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.769056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.769335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.783494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:11.789459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:11.790587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:11.791616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 407/805] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:11.792505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.806423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.807046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.807324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.807596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.809039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:11.809380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.810895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:11.816702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 399/806] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:11.817629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:11.940822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80432815-40d9-4a4d-a3c9-96f17256dbf1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:11.960087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:11.988364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:11.989489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:11.990115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:11.990487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:11.991417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:11.991819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:11.993606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.005292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 400/807] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:12.006246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.058787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec11f2-38ef-4b84-926f-e9c383a6db83 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.068672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.100663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.101331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.101673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.101956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.110771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.112383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:12.124781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:12.125736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:12.126654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 408/808] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:46:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:12.127228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.139472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec5c9d-d29f-40c1-b9ec-5facc4248a40 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.182420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.182797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.183435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.183718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.184390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:12.184699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.186071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.197773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 401/809] 10.222.0.57 () {56 vars in 1124 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:12.197773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.240578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.266112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef26c02-148d-47d2-91f3-714d0f93ec64 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.304867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.305536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.305819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.306086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.315858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc40c1f-24f0-46f1-bc94-daad108fd08f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.320829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.322305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:12.339282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:12.340595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:12.341478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:12.341478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:12.341478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:12.345834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 409/810] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:12.346827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.366948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.368106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.368634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.368787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.369536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.370984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:12.371445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.373113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.401419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 402/811] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:12.401419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.479655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcc12c-532b-4068-9cd3-c075300853dc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.505869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.510556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b8116-cce2-4a33-8111-f62046fe0ad5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.516297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.516924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.517209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.517479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.518132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.519504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.542355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 410/812] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:12.543375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.565951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.567006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.567508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.567988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.569060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:12.569609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.571228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0258c8-489d-4a73-90cc-3083ce929760 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.586179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 403/813] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:12.588403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a60d22-2b18-4fa4-9ecd-b1494ad1c22e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.588932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a60d22-2b18-4fa4-9ecd-b1494ad1c22e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.589442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a60d22-2b18-4fa4-9ecd-b1494ad1c22e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.591299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 404/814] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:12.597774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.598119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.598342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.622752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:12.624066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:12.659197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.694718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fdb74-5336-4558-8b9e-f4be5ebd6335 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.737464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 405/815] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:12.737464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.740682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.741358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.741643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.741916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.835657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.837382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.865906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.887922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e6aa6-20d9-447d-93f4-d9bef4e6de64 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:12.935912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:12.937732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:12.939399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:12.939996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:12.941560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:12.951233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:12.953779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:12.967905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 411/816] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:12.970091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:12.970994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:12.972512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:12.973050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:12.978584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 406/817] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:12.985244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.099151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b55b-e75d-4a2e-affb-0586107150cc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.117318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252631ce-9d35-47df-b051-b2ac71a1c9fa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.144186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.144856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.145137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.145409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.155516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.156485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.156966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.157505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.157738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.158622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:13.159131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.159233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.160937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:13.177785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:13.178755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:13.179623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:13.179623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:13.179623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:13.183691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 412/818] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:13.184592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.189797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 407/819] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Sep 15 12:46:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:13.191044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.194320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:13.218376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:13.302794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19281996-08e5-4b09-9dba-90ea92a0ce5d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.314010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e21cb2-4201-4add-824e-cc6d1e8b2f9d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.348075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.348769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.349134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.349473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.350113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.350233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.351008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.351459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.351747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.352103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.352934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:13.353405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.356438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:13.361933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:13.363006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:46:13.363845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:13.368085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 413/820] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:13.369015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.373477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 408/821] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:13.375477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.468663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:13.473197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:13.484510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fa237-fd26-4a7b-8f96-3200a27dc67f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.497257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caf142-9eb9-4e19-b414-058a7e0e2b2d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.520847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.521570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.521927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.522275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.533470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.535008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.536868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.537765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.538218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.538683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.539475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:13.539949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.541500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.547425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 409/822] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:13.548462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:13.555604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:13.556563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:13.557626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:13.557626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:13.557626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:13.557626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 414/823] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:13.563253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.663953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dc920-0947-45e1-bff5-10b2eef15997 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.673866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573c2ad-5628-4851-8456-89c7af03c852 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.710461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:13.716673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.717410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.717797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.717967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.718280 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.718886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.719093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:13.719463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.719593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.719987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.721120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.728340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.730001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.731230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 415/824] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:13.732397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.742714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:13.747749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:13.748983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:13.750058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:13.752503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 410/825] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:13.753469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.854971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a4834-a483-4cbf-880c-b7845c192d76 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.864088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514be065-e095-4e65-b291-f67d497d9378 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:13.894323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.894855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.895138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.895411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.896043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:13.896375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.897739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.901416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495bf67-de96-4b2a-a42e-085c3b9396e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:13.902334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495bf67-de96-4b2a-a42e-085c3b9396e5 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.902676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495bf67-de96-4b2a-a42e-085c3b9396e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.902951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495bf67-de96-4b2a-a42e-085c3b9396e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.903610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495bf67-de96-4b2a-a42e-085c3b9396e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:13.905434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495bf67-de96-4b2a-a42e-085c3b9396e5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:13.914587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 416/826] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:13.914587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.936069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 411/827] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:13.937512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9922b-e305-458e-a96b-6ca2ec97506a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:13.937765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9922b-e305-458e-a96b-6ca2ec97506a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:13.938128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9922b-e305-458e-a96b-6ca2ec97506a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:13.939542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 412/828] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:13.944776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9922b-e305-458e-a96b-6ca2ec97506a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:13.998333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:14.023112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:14.075201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9922b-e305-458e-a96b-6ca2ec97506a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.085952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3592d5-8f21-4488-b46a-ee8b5c007a46 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.123904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.124716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.125093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.125441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.133845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:14.135294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:14.138018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.138625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.138938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.139178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.139850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:14.140154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:14.141620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-39e2e73b-c43a-4767-a688-21d107295580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:14.147344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:14.148222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:14.149219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 413/829] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:14.150208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.150208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.150443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.162812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 417/830] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:14.163684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:14.171555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:14.172701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:14.249110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 414/831] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:14.250307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:14.288963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34ddfb-041b-498a-aa34-735a227f752f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.291419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:14.317846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:14.328065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.328993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.329479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.329981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.330870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:14.331331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:14.332953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:14.346143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 418/832] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:14.351709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:14.382385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f582c55-3f2b-4a2d-b5a0-5c40220f4cbe None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.428778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.430275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.430874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.431429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.432354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=03f7ea8f3fb64e3d96fbe277e07b5435) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:14.433554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:14.436203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:14.449154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 415/833] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:14 2022] GET /identity/v3/role_assignments?user.id=03f7ea8f3fb64e3d96fbe277e07b5435&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:14.476577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5a1e-5661-4226-a96c-e8fcb5d31a28 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.515937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.516651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.516933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.517208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/61362637fe434cbeba9d2d9efe37605a/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.534324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=61362637fe434cbeba9d2d9efe37605a, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:14.535773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:14.538347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:14.560916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 419/834] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:46:14 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/61362637fe434cbeba9d2d9efe37605a/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:46:14.662558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:14.674377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbf546-097e-4301-be5d-4f9677e460c9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.719132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97356318-b667-4430-8f80-340f8a85772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.720102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97356318-b667-4430-8f80-340f8a85772b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.720438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97356318-b667-4430-8f80-340f8a85772b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.720715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97356318-b667-4430-8f80-340f8a85772b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.734141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97356318-b667-4430-8f80-340f8a85772b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-97356318-b667-4430-8f80-340f8a85772b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:14.740555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:14.741541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:14.742361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 416/835] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:14.748082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:14.788729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:14.861523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15c13-a889-41d8-b5e1-2d1bef64847c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:14.899122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:14.899851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:14.900199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:14.900570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:14.901328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:14.902316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:14.903818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f518c-a096-4a6d-9aab-0bb72da030df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:14.914156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 420/836] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:14.920358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97356318-b667-4430-8f80-340f8a85772b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.018718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950028-ebe7-4498-b2b7-8748be8d7c98 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.018902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950028-ebe7-4498-b2b7-8748be8d7c98 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.019182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f950028-ebe7-4498-b2b7-8748be8d7c98 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.020736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 421/837] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:15.025726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.026006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.026279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.035105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97356318-b667-4430-8f80-340f8a85772b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.045379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:15.046565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:15.071196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.072072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.072535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.072959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.088096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.090431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:15.109361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:15.110545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:15.111620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:15.111620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:15.111620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:15.113377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 417/838] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:15.122365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.127380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:15.144326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 422/839] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:15.238906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4898e-c3e7-47d9-bc7c-f7244ee79321 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.278978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.279902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.280405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.280877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.281799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:15.282327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.284223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.296999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 418/840] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:15.302377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.333906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:15.412859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd7c24-0c06-46dd-a765-688346932377 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.432388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.449502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.450155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.450417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.450681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.458738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.460179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:15.473481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:15.474477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 423/841] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:15.480873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.507490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:15.553724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516563dd-621d-49c0-af1d-b13555ca145f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.588420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:15.596995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.597915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.598453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.599027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.599927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.601578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.602713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84033e-8a8e-4735-9637-e3ed59b31b9c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:15.612344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:15.613337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:15.614186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:15.621765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 419/842] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:15.622822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.638670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.639277 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.639555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.639832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.640571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:15.640872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.642858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.666875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 424/843] 10.222.0.57 () {56 vars in 1135 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:15.668017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.745439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216e8ed-c6ff-4d8d-8424-f9241ec2a906 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.775794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:15.783013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c719e-515f-49b3-a946-d7a3917aa3ef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.787191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.788017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.788489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.788925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.789697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:15.790145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.791829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.798291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 420/844] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:15.803607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:15.828616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.829856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.830365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.830868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.850833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:15.924996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.926623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.931463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb4e47-9628-435e-95e0-7461be56d647 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:15.978678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:15.979606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:15.980850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:15.981685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:15.982523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:15.983005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:15.985211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:15.992364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 421/845] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:16.045674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 425/846] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:46:16.051151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:16.233326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:16.281283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e2604-228f-4d84-9b1b-eae73bbd3049 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:16.317861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:16.318851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:16.319353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:16.319822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:16.331654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:16.333389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:16.349475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:16.350493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:16.351385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:16.351385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:16.351385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:16.353767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 422/847] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:16.362606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:16.413810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:16.470861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626bb01-b743-4e12-acb8-6f4989139a5f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:16.511142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:16.512085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:16.512397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:16.512674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:16.513310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:16.513612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:16.514970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:16.526940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 426/848] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:16.532352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:16.643227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eca56d-c635-48ef-a55e-dea32a39f1cb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:16.653260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:16.685142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:16.685828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:16.686108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:16.686387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:16.693722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:16.695298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a1076c17-51f1-4a67-823c-2f862df267cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:16.707089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:16.707973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:16.708899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 423/849] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:16.714882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:16.834162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55071fc-c1e7-4799-95bb-33bf9509a174 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:16.842343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:16.872423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:16.873030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:16.873312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:16.874584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:16.874584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:16.874584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:16.875954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:16.889870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 427/850] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:16.895802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1e02-f701-477b-989a-a3cf3eb46742 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:16.896259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1e02-f701-477b-989a-a3cf3eb46742 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:16.896259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1e02-f701-477b-989a-a3cf3eb46742 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:16.898090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:16.898212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 424/851] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:16.905415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252082a5-a000-4e8e-b43e-bccfce5c607a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:16.905701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252082a5-a000-4e8e-b43e-bccfce5c607a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:16.905982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252082a5-a000-4e8e-b43e-bccfce5c607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:16.931262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-252082a5-a000-4e8e-b43e-bccfce5c607a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:16.932659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-252082a5-a000-4e8e-b43e-bccfce5c607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:17.004986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 425/852] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:17.006607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0e956-98f6-462c-aa91-886c330348ee None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.006893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0e956-98f6-462c-aa91-886c330348ee None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.007162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0e956-98f6-462c-aa91-886c330348ee None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.013300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 426/853] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:17.014553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.014832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.015099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.025062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6256c7-3737-4ea8-844c-18208388bd47 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.037805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:17.039508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:17.073003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.074600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.075065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.075527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.076608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=4ead9001bd3241399dac6579a9588975) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:17.077887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:17.079528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:17.086143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 428/854] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:16 2022] GET /identity/v3/role_assignments?user.id=4ead9001bd3241399dac6579a9588975&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:17.119271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 427/855] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:17.276098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:17.310639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:17.388979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630dfd6-8091-4a4f-b972-709b058359b9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.396549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:17.425414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.426052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.426332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.426608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.427184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:17.428584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:17.431029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:17.439483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:17.440462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:17.441362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 429/856] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Sep 15 12:46:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:17.446565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:17.528768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93160e3-89fe-4b1b-80ab-e94ecc55951f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.550739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:17.558085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1d1f4-d85a-42f8-8973-81f6da893071 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.567574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.568531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.568964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.569420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.581009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:17.586331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:17.587433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:46:17.588435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:17.591766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 428/857] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:17.597580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:17.599881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.600524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.600805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.601074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.601699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:17.602001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:17.603351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:17.612305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 430/858] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:17.614890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:17.715342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:17.731982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4bb0b6-b73a-4c1c-a808-beca830781f0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.751398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6487b22-9d26-4ff7-a7c4-8850490f6cf4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.770128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.770820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.771160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.771703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.772300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:17.772677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:17.774110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:17.787408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 429/859] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:17.788354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:17.796116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.797010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.797474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.797951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.799213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:17.799707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:17.801634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:17.816302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 431/860] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:17.919092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93add8-77d1-4d65-8e6f-450823f93a5f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:17.958368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:17.959275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:17.959708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:17.960131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:17.972794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:17.974614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:17.991277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:17.992536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:17.993651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:17.993651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:17.993651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:17.996086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 430/861] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:18.002127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:18.033273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:18.125638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdf093-97ac-43b4-8f4d-dbab98b0188f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:18.162279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:18.162894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.163174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.163451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.164095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:18.164434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:18.165801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47ba46-4b60-4506-8352-57c7fe1868c8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:18.179176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 432/862] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:18.184767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:18.187949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:18.296154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b0e39-f825-4aa3-ae17-4f75cd5a1018 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:18.320721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a11ca-9ab9-42d7-b12f-0e0a302fb327 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.321185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a11ca-9ab9-42d7-b12f-0e0a302fb327 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.321462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a11ca-9ab9-42d7-b12f-0e0a302fb327 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.324642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 433/863] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:18.329577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.329863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.330143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.336728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:18.337353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.337630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.337902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.346784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:18.348169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:18.351572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:18.352929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:18.360593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:18.361517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:18.362445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 431/864] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:46:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:18.369344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:18.432374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 434/865] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:18.442642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:18.486279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f17805-ce63-4fb3-ac89-9b98958fe3b3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:18.521375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:18.521970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.522245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.522510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.523280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:18.523581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:18.525014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:18.544949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 432/866] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Sep 15 12:46:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:18.553411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:18.569052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:18.657680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075408f-36af-4919-9129-b6da7dc9e8f0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:18.699411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:18.700029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.700338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.700619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.742570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:18.777163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:18.778755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:18.797932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:18.864681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2687-b82e-47c3-80ec-003393ba7728 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:18.885884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:18.900965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:18.901720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:18.902044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:18.902319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:18.902988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:18.904394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:18.911845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 435/867] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:18.914775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:18.920815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 433/868] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:18 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:19.031594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7902e-2dfb-4d5b-8fce-afb4f0630e99 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:19.047507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:19.079864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:19.086543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.087336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.087849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.111099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:19.112897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:19.129591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:19.130599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:19.131489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:19.131489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:19.131489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:19.132081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 436/869] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:19.139465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:19.250815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bd14c-c49a-49da-a78a-3a5b0db910bb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:19.290571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:19.291194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.291475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.291749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.292521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:19.292824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:19.294193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9b99f-a886-4854-bbcc-d3066032394e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:19.307130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 434/870] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:19.313413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:19.367353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01d30-1415-4304-9049-9b9c9fd8bcde None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.367690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01d30-1415-4304-9049-9b9c9fd8bcde None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.367915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01d30-1415-4304-9049-9b9c9fd8bcde None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.369537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 435/871] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:19.374764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.375050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.375329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.394013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:19.395372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:19.428487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde1e9c-d5c4-4606-91dc-b6473952cf6d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:19.431701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:19.468309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 436/872] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:19.471782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:19.472704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.473183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.473649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.481247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:19.482935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:19.494508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:19.495954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:19.496863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:19.499167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 437/873] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:19.505135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:19.610609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ca926-0beb-4ae6-9574-6732fe6a08ba None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:19.646976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:19.647946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.648456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.648923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.649807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:19.650304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:19.652656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:19.665378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 437/874] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:19.671532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:19.695799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:19.758196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:19.788883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c3260-7599-44f1-89f2-21ad1a6e6e0c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:19.794792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:19.832430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:19.833200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.833689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.834066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.834934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=61362637fe434cbeba9d2d9efe37605a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:19.835411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:19.837392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:19.845354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 438/875] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:19 2022] GET /identity/v3/role_assignments?user.id=61362637fe434cbeba9d2d9efe37605a&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:19.881697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e856-ad2c-4715-8d3a-faf27856f15b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:19.921073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:19.922478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:19.922915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:19.923345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:19.924191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:19.925942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:19.936630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:19.937923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:19.938961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:19.945558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 438/876] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:19.948637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:20.046350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:20.068152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb351ddc-4a1c-4828-9764-4d9fd6c0e63f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:20.119742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:20.120427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:20.120714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:20.120993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:20.121620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:20.121928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:20.123290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db4f4c-4dc4-4f77-87b8-34b89378652f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:20.130497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 439/877] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:20.137580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:20.193763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc66f-66f8-4517-b271-eaf72a8a9c8c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:20.194088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc66f-66f8-4517-b271-eaf72a8a9c8c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:20.194322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc66f-66f8-4517-b271-eaf72a8a9c8c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:20.195763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 440/878] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:20.202101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152080f9-128f-4d4d-848c-0ce9148727cf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:20.202456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152080f9-128f-4d4d-848c-0ce9148727cf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:20.202800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152080f9-128f-4d4d-848c-0ce9148727cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:20.224091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-152080f9-128f-4d4d-848c-0ce9148727cf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:20.225948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-152080f9-128f-4d4d-848c-0ce9148727cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:20.245010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:20.252592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63dcd3-5f83-435e-8b6a-5ef9db7eb581 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:20.291223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:20.292397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:20.293168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:20.293666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:20.294489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:20.295184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:20.297281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:20.307009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 439/879] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:20.311058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 441/880] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:20.585892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:20.692870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2388ad-d373-479a-80e8-ff8b3a2aefd7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:20.729857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e0ca-ad62-4deb-bdc2-4e770aa926f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:20.730521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e0ca-ad62-4deb-bdc2-4e770aa926f7 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:20.730812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e0ca-ad62-4deb-bdc2-4e770aa926f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:20.731073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e0ca-ad62-4deb-bdc2-4e770aa926f7 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:20.731732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e0ca-ad62-4deb-bdc2-4e770aa926f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:20.733124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e0ca-ad62-4deb-bdc2-4e770aa926f7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:20.752303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 440/881] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:20 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:21.319119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30f8d-1f84-4703-b5ce-76309be31663 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:21.319670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30f8d-1f84-4703-b5ce-76309be31663 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:21.319670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd30f8d-1f84-4703-b5ce-76309be31663 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:21.321263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 442/882] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:21.326866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:21.327286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:21.327439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:21.346721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:21.347819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:21.361823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:21.424621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 441/883] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:21.702906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30f8d-1f84-4703-b5ce-76309be31663 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:21.785175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:21.814203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd30f8d-1f84-4703-b5ce-76309be31663 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:21.849153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:21.850087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:21.850563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:21.851024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:21.851780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:21.853368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:21.863243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:21.864334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:21.865238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:21.867572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 443/884] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:21.874237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:21.979048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e321e-6eff-44c5-ab46-a68d92b1f0e0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:22.022810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:22.023478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.023758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.024023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.024681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:22.025038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:22.027432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac41473-8724-418d-acc8-746e68a8fb79 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:22.034451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 442/885] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:22.041604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:22.189526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba8ce1-d610-4b53-9661-b2e431e5e9d1 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.189751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cca37-b9ac-45ff-b950-65d046ba23c1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:22.189988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba8ce1-d610-4b53-9661-b2e431e5e9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.190210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba8ce1-d610-4b53-9661-b2e431e5e9d1 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.192264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 443/886] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:22.197442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.197732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.198012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.219541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:22.221398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:22.223794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:22.234222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:22.234762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.235046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.235320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.235953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:22.236261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:22.237655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:22.247515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 444/887] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:22.249251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:22.261065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:22.332616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 444/888] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:22.376670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b8e6f-c687-4252-a7be-113c336ed0ae None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:22.419519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:22.420139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.420466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.420745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.421444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] RBAC: Adding query filter params (interface=public, service_id=e735550704964cd49caa6abaabdfab9a, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:22.421758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:22.423130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:22.429094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 445/889] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:22 2022] GET /identity/v3/endpoints?service_id=e735550704964cd49caa6abaabdfab9a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:22.434975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:22.546256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc51d3-3bfa-495a-a1f3-7568646feb62 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:22.586631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a631ca0-c211-4a07-8145-184feceb0aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:22.587828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631ca0-c211-4a07-8145-184feceb0aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.588371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631ca0-c211-4a07-8145-184feceb0aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.588889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a631ca0-c211-4a07-8145-184feceb0aa5 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.590269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a631ca0-c211-4a07-8145-184feceb0aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:22.591995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a631ca0-c211-4a07-8145-184feceb0aa5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:22.599529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 445/890] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:22 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:46:22.631425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:22.720869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:22.739899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e47269-5080-428b-9d8d-e6c03615bf47 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.740577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e47269-5080-428b-9d8d-e6c03615bf47 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.741167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e47269-5080-428b-9d8d-e6c03615bf47 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.743024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 446/891] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:22.752156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.752808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.753276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.760480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:22.763465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348fa81-aff5-45f2-8f88-57c13370ce18 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:22.776564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:22.778353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:22.810942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:22.811508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.811782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.812075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.826478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:22.831692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:22.832784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:22.833694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 446/892] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Sep 15 12:46:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:22.839141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:22.867082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 447/893] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:22.959578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cf38d-5cb3-47e1-83c0-0f229d7e7bef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:22.991107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:22.996684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:22.997516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:22.997948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:22.998384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:22.999192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:22.999662 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.001213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.010729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 447/894] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:46:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:23.017607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.126303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ea556-7e83-4443-822f-fb38c2b56eab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.157104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:23.162266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.162927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.163171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.163445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.164947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.178624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.180769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:23.201762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:23.202778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:23.203703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:23.203703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:23.203703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:23.203703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 448/895] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:23.209776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.275527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab7ba2-1555-4596-abaf-214922cca79f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.313660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e740f-fa63-4e06-b726-c863d6a0d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.314298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e740f-fa63-4e06-b726-c863d6a0d3f5 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.314712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e740f-fa63-4e06-b726-c863d6a0d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.314998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e740f-fa63-4e06-b726-c863d6a0d3f5 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.315945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e740f-fa63-4e06-b726-c863d6a0d3f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.317359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e740f-fa63-4e06-b726-c863d6a0d3f5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.318994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdbe5d-1846-4ec3-8aa2-bc5eac8698a2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.333373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 448/896] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:23 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:23.345806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f556e6b-298d-4adc-9c5e-01a04b37b6c3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.346146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f556e6b-298d-4adc-9c5e-01a04b37b6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.346376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f556e6b-298d-4adc-9c5e-01a04b37b6c3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.352041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 449/897] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:23 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:23.358495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.359169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.359884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.368096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.368743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.369027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.369300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.369952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:23.370832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.374187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.388535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:23.389765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:23.394553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 449/898] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:23.400010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.455113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:23.464573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:23.496487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 450/899] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:23.536698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761864d4-44f5-4bf4-b946-e0205424586d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.571537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.572163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.572470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.572741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.580176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.582280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:23.597643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:23.598831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:23.599960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 450/900] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:23.605520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.715923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba71cf2-33d7-437c-9822-7792b06f424f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.752184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.752886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.753164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.753435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.754211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:23.754507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.755933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.775085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 451/901] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:23.777390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.780704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:23.824682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:23.890621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749cee4-90e2-4e27-9d23-69e52fb789e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.896784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:23.899925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255ba86-e131-48cc-bdeb-3549f61d8ef8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:23.931292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.932295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.933395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.934069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.938724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:23.939629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:23.940245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:23.940877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:23.942058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:23.944812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:23.955998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:23.957214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:23.958297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:23.960315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 451/902] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:23.966157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.019564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.021166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.066651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:24.115982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d86e3-2de9-46d3-b4ef-cb8f3a0b4cdc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.147806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 452/903] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:24.149374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.159761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.160812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.161287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.161839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.162876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:24.163401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.165179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.171661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 452/904] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:24.178595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.214584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:24.280786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c4a1e-8eee-43ab-9532-afc70ab3d288 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.302871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090131cb-14ea-43c0-8251-1d2c596ae9b4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.324609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.325260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.325543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.325818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.336796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:24.340533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.342636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.346786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.347678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.348161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.348673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.349528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:24.350036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.352386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.360105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 453/905] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:24.365332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:24.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:24.367210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:24.367210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:24.367210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:24.367907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 453/906] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:24.369848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.372655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.488741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:24.502218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f64ac-fa76-4ec6-bf6c-f4471040ba5e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.503950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623852e-f9be-4400-828a-0a8bc2aaa8aa None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.547213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.548110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.548585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.549008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.549845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:24.550303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.551638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.552083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.552387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.552740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.553129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.553903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] RBAC: Adding query filter params (interface=public, service_id=e7a3d771cf6840b9b2b69951cd2b888b, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:24.554327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.556604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.564428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 454/907] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/endpoints?service_id=e7a3d771cf6840b9b2b69951cd2b888b&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:24.568493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.576621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 454/908] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:24.577996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.640002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:24.701668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec84ade-8386-4803-ae83-c9637a458041 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.702878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5af7b-0b3d-47e6-ad1d-18ba8eb6a994 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.738400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.739188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.739471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.739813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.746908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.748479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.751400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.752219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.752726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.753186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.753983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.755785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:24.760409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:24.762091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:24.762959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 455/909] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:24.772562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 455/910] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/services => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:46:24.773719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.896243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971d628-2e99-4873-865d-afa9af4b54f1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:24.911618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:24.937061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:24.938003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.938466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.938920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.939830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:24.940346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:24.942028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3455b-8b3b-45e6-8cca-9f08b99df676 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:24.957288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 456/911] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:24.964414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:24.973178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f6336-5b92-4188-914e-35a4c758f7fc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.973470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f6336-5b92-4188-914e-35a4c758f7fc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.973777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f6336-5b92-4188-914e-35a4c758f7fc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.975225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 457/912] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:24.981275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:24.981565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:24.981961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:24.994004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:25.003385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:25.004656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:25.088010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 458/913] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:25.098501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182b0fb-a398-40c4-81e2-3a68d76c8253 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:25.137224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:25.138083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:25.138533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:25.138972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:25.139902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] RBAC: Adding query filter params (role.id=e0c7d64dc2c949c0972dcbc6b8712ed6, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=61362637fe434cbeba9d2d9efe37605a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:25.140404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:25.142037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:25.152407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 456/914] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:24 2022] GET /identity/v3/role_assignments?user.id=61362637fe434cbeba9d2d9efe37605a&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=e0c7d64dc2c949c0972dcbc6b8712ed6 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:25.359354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:25.464567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864d195-58c3-4023-aa4f-d11e155c0817 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:25.466227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:25.500140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:25.500815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:25.501099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:25.501372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:25.501948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:25.503315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-864b0dd1-a017-482d-931d-e61ee45fadd6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:25.514070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:25.515499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:25.516802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 459/915] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:25.523221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:25.621685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:25.640634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6cd2b-82a9-49da-8ff4-09fe4de197e5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:25.680186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:25.680817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:25.681096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:25.681362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:25.681988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:25.682308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:25.682420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994398e-e804-4ea0-9045-df119b4ab7d1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:25.682767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994398e-e804-4ea0-9045-df119b4ab7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:25.683006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994398e-e804-4ea0-9045-df119b4ab7d1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:25.684052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:25.684834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 460/916] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:25.689540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 457/917] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:25.693741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:25.694294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:25.694926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:25.695061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:25.716691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:25.718215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:25.797798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 461/918] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:25.815639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595647b-b5d0-492c-89ff-99dd4d1a9b4e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:25.852670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:25.853274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:25.853548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:25.853817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:25.854640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:25.855179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:25.856730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:25.862239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 458/919] 10.222.0.57 () {56 vars in 1114 bytes} [Thu Sep 15 12:46:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:25.867816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:25.968239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:25.974919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd9aed-dcb3-4bc6-ae8c-0ddf57603d91 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:26.014196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:26.015041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:26.015473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:26.015926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:26.016796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:26.018307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:26.040354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 462/920] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:26.084944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:26.106576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:26.202861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080d014-c4bc-4122-95c8-391a795613c6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:26.246704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:26.247251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:26.247526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:26.247794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:26.248401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:26.250697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:26.260436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:26.262544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:26.264144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 459/921] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:26.268114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:26.375785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc471d-3e81-49ee-a90c-f167c1571ba6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:26.411826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:26.412471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:26.412761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:26.413037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:26.413666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:26.413973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:26.415339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:26.421200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 463/922] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:26.426513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:26.497820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:26.533233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a85d7-dc95-4191-9852-b0ee4175eaf4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:26.568470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:26.569077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:26.569355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:26.569625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:26.570246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:26.570543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:26.571905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:26.577666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 460/923] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:26.583475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:26.603528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:26.697353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d01f92-84b4-4d39-8459-2f9d2837719e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:26.732104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:26.732741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:26.733019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:26.733291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:26.733980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] RBAC: Adding query filter params (interface=public, service_id=31b4611c4fbe41a4ab3b5cd20326fac0, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:26.734285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:26.735642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3de3bf-97b4-458d-8f41-dd2fd80cbfce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:26.741030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 464/924] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:26 2022] GET /identity/v3/endpoints?service_id=31b4611c4fbe41a4ab3b5cd20326fac0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:26.747445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:26.856848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6481e-dd39-42d5-8a4b-04e03f747f5e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:26.892886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc2cb9-f50b-4a35-8ef8-618c092cb07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:26.893495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc2cb9-f50b-4a35-8ef8-618c092cb07e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:26.893773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc2cb9-f50b-4a35-8ef8-618c092cb07e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:26.894039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc2cb9-f50b-4a35-8ef8-618c092cb07e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:26.894641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc2cb9-f50b-4a35-8ef8-618c092cb07e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:26.896005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc2cb9-f50b-4a35-8ef8-618c092cb07e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:26.904010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:26.904258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 461/925] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:26 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:46:27.200623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f76f3d-4f7f-47b0-a308-c656f46ba582 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:27.200814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f76f3d-4f7f-47b0-a308-c656f46ba582 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:27.201208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f76f3d-4f7f-47b0-a308-c656f46ba582 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:27.202894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 465/926] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:27.208375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fbac9-2360-4d5e-b070-79d20918f867 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:27.208745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fbac9-2360-4d5e-b070-79d20918f867 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:27.209097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fbac9-2360-4d5e-b070-79d20918f867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:27.227680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7a5fbac9-2360-4d5e-b070-79d20918f867 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:27.228890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a5fbac9-2360-4d5e-b070-79d20918f867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:27.299007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 462/927] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:27.452544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca814c59-d32c-4e1c-8f14-77b74fe81f79 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:27.452732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca814c59-d32c-4e1c-8f14-77b74fe81f79 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:27.453001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca814c59-d32c-4e1c-8f14-77b74fe81f79 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:27.454649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 466/928] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:27.459837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:27.460141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:27.460446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:27.479429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:27.481395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:27.551851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 463/929] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:27.578920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca814c59-d32c-4e1c-8f14-77b74fe81f79 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:27.676992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:27.685541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca814c59-d32c-4e1c-8f14-77b74fe81f79 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:27.723055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:27.723691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:27.723968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:27.724237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:27.724861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:27.726214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:27.735218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:27.736149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:27.737079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 467/930] 10.222.0.57 () {56 vars in 1132 bytes} [Thu Sep 15 12:46:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:27.742272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:27.803494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:27.845524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda02b4-b020-459e-bbe8-5545aabd70e8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:27.856793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:27.889868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:27.890492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:27.890774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:27.891048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:27.891680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:27.891985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:27.893394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:27.900415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 464/931] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:27.905033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:27.977551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0822ff-77e1-4dc3-8c36-84dac0b421f4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.014019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.014652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.014929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.015199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.025626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:28.026351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda44268-b302-40e3-9f01-8fa2ed5727db None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:28.030435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:28.031357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.032193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 468/932] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Sep 15 12:46:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:28.038619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.043788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:28.059060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:28.074173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.074914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.075370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.075645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.076266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:28.076583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.078409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.095575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 465/933] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:28.098310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.174501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63650-49e3-45d8-b772-88dc16a5b873 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.219329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.220301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.220788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.220896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1e753-6d68-4f0f-92a8-a831dbc8f749 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.221337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.222193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:28.222828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.225230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.236240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 469/934] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:46:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:28.241966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.267008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c429a-2668-4c1e-8454-6035f0444316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.267708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c429a-2668-4c1e-8454-6035f0444316 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.267996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c429a-2668-4c1e-8454-6035f0444316 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.268291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c429a-2668-4c1e-8454-6035f0444316 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.269007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c429a-2668-4c1e-8454-6035f0444316 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.270640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c429a-2668-4c1e-8454-6035f0444316 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.304677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 466/935] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:28.326394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:28.368446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c686d-8cf6-4136-a28e-133365565dfc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.385930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e79f4-6ccc-4a0f-b024-4b36a4e4a6f0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.386382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e79f4-6ccc-4a0f-b024-4b36a4e4a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.386669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e79f4-6ccc-4a0f-b024-4b36a4e4a6f0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.392626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 467/936] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:28.394430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.394809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.395156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.414166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.414810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.415088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.415360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.416446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:28.418627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:28.428202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.429830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:28.446728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:28.447739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:28.448685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:28.448685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:28.448685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.448685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 470/937] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:28.455386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.476065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:28.516734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 468/938] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:28.585163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20e069-d75c-4846-8a8a-a88cf69716e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.600975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:28.623039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.623659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.623941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.624215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.624961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:28.625272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.626793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.640734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 471/939] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:28.646304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.757095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c1564-f27b-4754-a4f4-c5fceddff1d7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.786102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.796549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.797423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.797868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.798730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.809558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.811336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:28.828639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:28.830098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:28.831233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:28.833414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 469/940] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:46:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:28.839352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.876761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:28.910869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbde4e-d109-466c-bd47-6106381938e6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.950690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092ac4b-c975-4c03-9856-0ac6ceea5fe4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:28.951805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:28.952538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:28.953005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:28.953173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:28.953981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:28.955481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:28.958279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING keystone.server.flask.common [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 12:46:28.978472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 12:46:28.978472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 12:46:28.978472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [parameters: {'id': '40d4af8ed4d5460e9a6c90e77c1f3711', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 12:46:28.978472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 40d4af8ed4d5460e9a6c90e77c1f3711.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 40d4af8ed4d5460e9a6c90e77c1f3711. Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:46:28.987699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 15 12:46:28.988541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 15 12:46:28.989483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 12:46:28.990355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 12:46:28.991214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:46:28.991986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': '40d4af8ed4d5460e9a6c90e77c1f3711', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.992875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:28.993645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 40d4af8ed4d5460e9a6c90e77c1f3711. Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:28.996783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 472/941] 10.222.0.57 () {60 vars in 1140 bytes} [Thu Sep 15 12:46:28 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:28.998143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:28.999620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.000497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.000990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.001463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.002396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:29.002918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.004575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.028365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 470/942] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Sep 15 12:46:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:29.030018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.034688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:29.140215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43752cb8-76df-43c7-b55b-59bdeffad2d2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.160963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:29.171519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa951a84-a0bd-4db5-abda-c01359083234 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.186468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.187121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.187491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.188061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.196639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.199205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.210041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.210958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.211434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.211971 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3a325068-15f8-4531-9070-9abefa82592f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:29.212429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:29.213254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:29.214756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:29.218498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 473/943] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:29.220743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa1936-cd3d-493d-a8a6-a8865b7c70f7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.221037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa1936-cd3d-493d-a8a6-a8865b7c70f7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.221313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa1936-cd3d-493d-a8a6-a8865b7c70f7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.226059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 474/944] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:29.226988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa1936-cd3d-493d-a8a6-a8865b7c70f7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.229982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.232018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a97a0c30-8043-4036-9234-9d3725a8902a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:29.247162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:29.248119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:29.249048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:29.249048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:29.249048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:29.249048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 471/945] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:29.249966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.250172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.250513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.269334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:29.270568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:29.332904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:29.344781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 472/946] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:29.345979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.354172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa1936-cd3d-493d-a8a6-a8865b7c70f7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.401281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.402703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.403055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.403399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.404123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:29.404528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.406525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.430441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:29.430758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 475/947] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:29.465444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262c97-72ec-4d18-aaa7-35072b3ad676 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.506578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.507557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.508041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.508528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.509397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:29.509888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.511997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.526822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 473/948] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:29.532963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.643457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8885ec8-9223-4751-904d-d37e9f3a15f2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.648061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.691887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.692913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.693404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.693917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.702194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.703651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:29.715474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:29.716445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:29.717313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 476/949] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:29.722402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.735230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:29.785525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed70c3dc-0d55-4a22-a40e-29d67efc9280 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.822255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.823057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.823966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.824526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.825286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.826730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.833884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90ddbc-08b9-48f6-9ddb-0c03eaaf12be None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:29.836627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:29.837603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:29.838425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:29.843454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:29.843454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 474/950] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:29.844155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.870676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:29.871350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:29.871744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:29.872029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:29.872749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:29.873115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:29.874487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:29.891259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 477/951] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:29.897260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:29.965315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f94e4-750f-448f-99e2-307c4143989b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:29.974842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:30.006217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:30.007181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:30.007602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:30.008014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:30.008940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:30.009383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:30.010397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa596fc-57c5-4931-9eda-2af0e65987a8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:30.011204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:30.017400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 475/952] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:30.024389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:30.056978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:30.056978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:30.056978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:30.056978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/61362637fe434cbeba9d2d9efe37605a/roles/e0c7d64dc2c949c0972dcbc6b8712ed6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:30.060986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=61362637fe434cbeba9d2d9efe37605a, role_id=e0c7d64dc2c949c0972dcbc6b8712ed6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:30.062416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:30.084342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 478/953] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:46:29 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/61362637fe434cbeba9d2d9efe37605a/roles/e0c7d64dc2c949c0972dcbc6b8712ed6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:46:30.116592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:30.163331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972a040-b05a-4fd7-9c99-fc8384678e41 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:30.204411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:30.205039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:30.205320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:30.205597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:30.206237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:30.206542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:30.207981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b202-7184-4f9e-9492-0a74da409d42 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:30.214710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 476/954] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:30.220669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:30.340536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857941f-f911-4aab-a6c8-c11ae4f8e7df None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:30.373991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:30.378355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e942b73-e76a-43b8-a4fe-1c5f4f377171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:30.379228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e942b73-e76a-43b8-a4fe-1c5f4f377171 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:30.379938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e942b73-e76a-43b8-a4fe-1c5f4f377171 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:30.380313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e942b73-e76a-43b8-a4fe-1c5f4f377171 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:30.381060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e942b73-e76a-43b8-a4fe-1c5f4f377171 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:30.382495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e942b73-e76a-43b8-a4fe-1c5f4f377171 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:30.407776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 479/955] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:30.657653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573415d-2c58-4b8b-9a8a-0a435769e649 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:30.658178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573415d-2c58-4b8b-9a8a-0a435769e649 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:30.658178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573415d-2c58-4b8b-9a8a-0a435769e649 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:30.659741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 477/956] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:30.664964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:30.665253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:30.665527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:30.684720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:30.686222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:30.765565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 480/957] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:31.037428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573415d-2c58-4b8b-9a8a-0a435769e649 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:31.153158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573415d-2c58-4b8b-9a8a-0a435769e649 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:31.179618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:31.190747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:31.191694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:31.192172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:31.192666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:31.193519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:31.195224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:31.205526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:31.206632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:31.207494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:31.210306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 478/958] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:31.220339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:31.321989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4d0c2-c620-4406-87aa-1db904502421 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:31.337363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:31.361465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:31.362086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:31.362368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:31.362644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:31.363275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:31.363580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:31.364983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681c1e7-bd4a-4c74-a69d-84c317ef4816 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:31.370490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 481/959] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:31.375857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:31.481346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee06dc-f148-4c9f-9f95-cb73af52c7b1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:31.516891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:31.517492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:31.517820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:31.518096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:31.518718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:31.519063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:31.520475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdb07f-09ac-4cb8-b1a8-ff132e7f0f69 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:31.525300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 479/960] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:31.839690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524194-65c3-45d9-99e9-b6a31b6a2eed None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:31.839985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524194-65c3-45d9-99e9-b6a31b6a2eed None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:31.840281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524194-65c3-45d9-99e9-b6a31b6a2eed None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:31.841985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 482/961] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:31.847321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:31.848165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:31.848673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:31.867898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:31.869444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:31.928912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:31.947755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 480/962] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:32.224408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524194-65c3-45d9-99e9-b6a31b6a2eed None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:32.339573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524194-65c3-45d9-99e9-b6a31b6a2eed None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:32.379329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:32.379871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.380153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.380461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.381119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:32.382581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:32.398419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 12:46:32.398419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 12:46:32.398419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [parameters: {'user_id': 'd92bebf9e9714893a74b8c14e899bee2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 12:46:32.398419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2e02f118-22ac-4b4d-bf34-2f5af440421d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 15 12:46:32.405478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:46:32.406320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 12:46:32.407276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 12:46:32.408164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:46:32.409096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:46:32.409961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd92bebf9e9714893a74b8c14e899bee2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:32.410789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 15 12:46:32.411573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:32.416060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 483/963] 10.222.0.57 () {60 vars in 1131 bytes} [Thu Sep 15 12:46:32 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:32.416480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:32.496288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d905a-17f1-4f16-8946-11ed40d636bc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.496639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d905a-17f1-4f16-8946-11ed40d636bc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.496902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d905a-17f1-4f16-8946-11ed40d636bc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.501443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 484/964] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:32.506881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b0d86-7263-4b01-8766-6cc678106ecc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.507167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b0d86-7263-4b01-8766-6cc678106ecc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.507440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b0d86-7263-4b01-8766-6cc678106ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.530280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01eb3a7-2bcc-4dce-9d35-de2c28fc8530 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:32.530917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2c0b0d86-7263-4b01-8766-6cc678106ecc None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:32.532911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c0b0d86-7263-4b01-8766-6cc678106ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:32.569590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:32.570287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.570563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.570834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.574163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:32.579599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:32.581155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:32.593835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:32.594933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:46:32.595870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:32.599835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 481/965] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:32.601107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:32.634428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 485/966] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:32.661852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:32.670293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c551-5d12-484f-bf0a-8d05d74f60e0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.670579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c551-5d12-484f-bf0a-8d05d74f60e0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.670863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3c551-5d12-484f-bf0a-8d05d74f60e0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.672488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 486/967] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:32.677870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.678156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.678435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.702631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:32.704390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:32.730811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7116050-f35f-48ec-9ff7-7fc138496299 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:32.775773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:32.777008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:32.777291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:32.777564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:32.778532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:32.778842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:32.781046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:32.792827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 487/968] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:32.824454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 482/969] 10.222.0.57 () {56 vars in 1101 bytes} [Thu Sep 15 12:46:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:32.920447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.032825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.036854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce535fc0-aec3-467b-8cda-f8821ab97bd2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.066531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.075350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.075996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.076299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.076584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.091381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:33.096185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:33.097190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:33.098536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 488/970] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Sep 15 12:46:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:33.106846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.197041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35206adf-f315-4ed7-990e-60bed4e0d51d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.199368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.226202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629a1be-fa7f-4f15-ae71-18546007e705 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.237013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.237958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.238243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.238670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.239255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.240645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:33.251080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:33.252091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:33.252975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 483/971] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:33.257912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.266084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.266700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.266983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.267255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.267912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:33.268217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.269742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.282700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 489/972] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:33.283769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.306515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.369940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690a9d-587c-4128-b26d-4230e0fa26eb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.389199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13092f-ad72-4811-9538-669080cb94b8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.406319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.406932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.407213 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.407486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.408113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:33.408448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.409824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.414774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 484/973] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:33.420954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.428399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.429036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.429316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.429585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.440845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.442290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.453888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:33.457288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:33.458219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:33.459110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:33.459110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:33.459110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:33.459110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 490/974] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:33.464430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.534737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565225a3-c046-401e-b14c-bd8c2e81f8d9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.541292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.569309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.569921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.570198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.570468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.571094 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:33.571392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.572790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.577494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 485/975] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:33.578095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e96c8-d428-40f7-83a5-f3a2f9d417e3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.583057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.619176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.619994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.620323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.620602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.621233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:33.621538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.622897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.634762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 491/976] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:33.641415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.689076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.697333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600c1b-a6e9-40e9-ba65-0b16980124dc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.737733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.738260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.738542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.738820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.739532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] RBAC: Adding query filter params (interface=admin, service_id=31b4611c4fbe41a4ab3b5cd20326fac0, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:33.739844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.741253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.746961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 486/977] 10.222.0.57 () {56 vars in 1249 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/endpoints?service_id=31b4611c4fbe41a4ab3b5cd20326fac0&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:33.752397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.753148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d963b1-438d-4709-9dde-12bcab450b0d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.786664 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.793137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.793778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.794059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.794606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.802194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.803342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-681bba8a-989f-4afd-9b70-e833d7758b54 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:33.815251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:33.816223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 492/978] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021dd9d-e20f-4181-a764-fc8e0035c4d9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.822320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021dd9d-e20f-4181-a764-fc8e0035c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.824073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021dd9d-e20f-4181-a764-fc8e0035c4d9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.824073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 493/979] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:33.824427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021dd9d-e20f-4181-a764-fc8e0035c4d9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:33.865537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916341f7-ab85-4c3e-9a38-ee3f24ddbf45 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.899031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162600-563b-44c8-8173-8b39c944897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.899639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162600-563b-44c8-8173-8b39c944897b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.899918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162600-563b-44c8-8173-8b39c944897b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.900189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162600-563b-44c8-8173-8b39c944897b None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.900819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b162600-563b-44c8-8173-8b39c944897b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.902809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b162600-563b-44c8-8173-8b39c944897b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:33.909090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 487/980] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:46:33.913723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.914170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.914430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.934325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:33.935557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:33.936754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021dd9d-e20f-4181-a764-fc8e0035c4d9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:33.955664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:33.982785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:33.983418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:33.983696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:33.983964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:33.985396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:33.985686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:33.987572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:34.014765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 494/981] 10.222.0.57 () {56 vars in 1149 bytes} [Thu Sep 15 12:46:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:34.015873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:34.021012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 488/982] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:34.074387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:34.137897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893975b-72e4-44c1-afad-2fcef2240b3f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:34.172620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:34.173231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:34.173506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:34.173770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:34.242409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:34.244426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:34.294148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:34.300008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:34.356582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 495/983] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:46:34.362403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:34.410519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c137f4-d77f-496a-9744-49d8da5c2a7e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:34.446095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:34.446965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:34.447252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:34.448388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:34.449082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:34.450447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:34.466690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 489/984] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:34 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:34.470629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338e9a9-077e-40db-937b-63bdd6dfdb3c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:34.517810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:34.518459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:34.518741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:34.519017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:34.531622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:34.533118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:34.540966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:34.551560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:34.552818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:34.554279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:34.554279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:34.554279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:34.554279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 496/985] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:34.564310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:34.576953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:34.672309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdbd8a-b48a-4586-bdb0-1c3970cd2494 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:34.707221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:34.707826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:34.708104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:34.708396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:34.709158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:34.709465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:34.710823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:34.724519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 490/986] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:34.730208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:34.836012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3d700-881f-49d7-a87a-1251599f93bc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:34.871173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:34.871869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:34.872150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:34.872452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:34.879231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:34.880681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:34.893716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:34.894614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:34.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:34.898830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 497/987] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:34.901230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:34.974579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:35.005970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb81d73-e6ef-4c55-bbf8-bcd85e6a4a02 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:35.040878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:35.041481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:35.041784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:35.042053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:35.042896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:35.043198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:35.044650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781e5d9-d7a2-4e1e-ba6f-34aaf5bc2eaa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:35.056311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 491/988] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:35.062977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:35.132795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:35.176785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa735618-6819-41a9-9139-3e6c001cc927 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:35.211822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:35.212466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:35.212746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:35.213018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:35.213928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] RBAC: Adding query filter params (role.id=e0c7d64dc2c949c0972dcbc6b8712ed6, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=61362637fe434cbeba9d2d9efe37605a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:35.214319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:35.215745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f99649-b6a3-4850-9365-4880152da3cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:35.221968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 498/989] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:35 2022] GET /identity/v3/role_assignments?user.id=61362637fe434cbeba9d2d9efe37605a&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=e0c7d64dc2c949c0972dcbc6b8712ed6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:36.109450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ea7e7-a392-4fc2-ac0f-74f47781421e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.109969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ea7e7-a392-4fc2-ac0f-74f47781421e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.110047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ea7e7-a392-4fc2-ac0f-74f47781421e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.111561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 492/990] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:36.117285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.117815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.117893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.137736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:36.138930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:36.206742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 499/991] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:36.475990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ea7e7-a392-4fc2-ac0f-74f47781421e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:36.582944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ea7e7-a392-4fc2-ac0f-74f47781421e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:36.612732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:36.618806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:36.619439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.619763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.620062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.620666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:36.622037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e981d570-9804-41ec-a8ce-f621035c1317 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:36.630679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:36.631649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:46:36.632892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:36.636442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 493/992] 10.222.0.57 () {56 vars in 1126 bytes} [Thu Sep 15 12:46:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:36.638567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:36.676035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b64601-8a25-45fa-b645-de04d6454264 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.676399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b64601-8a25-45fa-b645-de04d6454264 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.676672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b64601-8a25-45fa-b645-de04d6454264 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.678137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 494/993] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:36.683738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.684252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.684751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.702605 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:36.704432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:36.706102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:36.750016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab7631-91b8-4875-ad8c-e59260bbbb38 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:36.778033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 495/994] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:36.784979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:36.785594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.785876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.786150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.786776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:36.787081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:36.788482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:36.798505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 500/995] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:36.799423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:36.897046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5655d-1743-4f93-854d-53462c61f00d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:36.924599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:36.934494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:36.935104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:36.935382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:36.935651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:36.936298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:36.936620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:36.937980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:36.947404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 496/996] 10.222.0.57 () {56 vars in 1122 bytes} [Thu Sep 15 12:46:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:36.949075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.047142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.050338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e188-0d3c-4c7b-aacd-9642a93c3d88 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.068999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:37.093905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.094523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.094811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.095078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.095745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.097705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.113501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 501/997] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:37.169548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cabbd-7694-4996-bf56-8620f00b2126 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.218224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.219274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.219758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.220226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.221067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.222696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-05c4c23b-a521-4d80-9b55-2a86ff12032d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:37.233044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:37.234098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:37.234969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:37.237278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 497/998] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:37.243289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.360614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a3f73-7d79-42d9-8fa5-85b65586f450 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.395892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.396534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.396822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.397094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.397716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:37.398019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.399408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.405162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 502/999] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:37.406455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487207-0d5f-473a-bd8c-96fd6c207803 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.406749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487207-0d5f-473a-bd8c-96fd6c207803 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.407025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487207-0d5f-473a-bd8c-96fd6c207803 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.409537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 498/1000] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:37.410600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.413550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.413837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.414998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.434900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:37.436096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:37.459868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:37.486279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:37.508738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 499/1001] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:37.529193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf053de-075c-4fa4-aadf-d766ee05b947 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.563188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.563801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.564082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.564379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.564997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:37.565299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.566661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.576810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 503/1002] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:37.578327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.680801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9975d00-a17c-4ad8-b424-f2ba9a68b396 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.716054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.716696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.717009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.717293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.717989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] RBAC: Adding query filter params (interface=public, service_id=8316a2f5875f4856ad780e39e7111d92, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:37.718345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.719716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.729241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 500/1003] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/endpoints?service_id=8316a2f5875f4856ad780e39e7111d92&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:37.730829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.778243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:37.828809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:37.839060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881f0d3-87eb-43ea-8997-93f097f360d9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.843552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:37.878581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.879196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.879476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.879843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.880379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.881950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.900303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 504/1004] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:46:37.900626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c33a58-74c1-4b3b-8826-decf75857588 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:37.939733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:37.940510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:37.940941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:37.941342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] PATH_INFO: `/v3/services/image` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:37.942246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:37.944214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:37.955242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:37.960546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:37.961451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 501/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:37.962677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:38.086589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d950757-f294-4414-a647-649fff56fdcc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:38.124347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:38.125283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:38.125764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:38.126251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:38.127084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:38.127595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:38.129203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85d2da-8101-4369-96e5-c1dc607a1fe3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:38.134577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 505/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:38.140956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:38.241306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f6bca-29d5-4945-80a7-8d2f6f869adf None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:38.250584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:38.278068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:38.278677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:38.278955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:38.279226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:38.279858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] RBAC: Adding query filter params (type=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:38.280156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:38.281562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd46234-b8fb-42b9-9114-6ffbb7e93565 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:38.286289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 502/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:39.348258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6bde2-f3ea-4aa8-80d3-9447701f22d5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:39.348767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6bde2-f3ea-4aa8-80d3-9447701f22d5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:39.348767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6bde2-f3ea-4aa8-80d3-9447701f22d5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:39.350261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 506/1008] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:39.355734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:39.356565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:39.357082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:39.379081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:39.380811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:39.456451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 503/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:39.721930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6bde2-f3ea-4aa8-80d3-9447701f22d5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:39.755033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:39.829959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6bde2-f3ea-4aa8-80d3-9447701f22d5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:39.864588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be359972-264f-40c2-a6fe-8e4212c6e099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:39.865229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359972-264f-40c2-a6fe-8e4212c6e099 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:39.865513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359972-264f-40c2-a6fe-8e4212c6e099 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:39.865793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be359972-264f-40c2-a6fe-8e4212c6e099 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:39.867183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359972-264f-40c2-a6fe-8e4212c6e099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:39.868689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be359972-264f-40c2-a6fe-8e4212c6e099 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:39.871127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 507/1010] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:39.876667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:39.977409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb6fc9-365d-47a2-8e26-ce197402e535 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:39.978886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:40.015558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902255c7-a37c-4463-aecb-95985bdee890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:40.016200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902255c7-a37c-4463-aecb-95985bdee890 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.016515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902255c7-a37c-4463-aecb-95985bdee890 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.016790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902255c7-a37c-4463-aecb-95985bdee890 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.017628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902255c7-a37c-4463-aecb-95985bdee890 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:40.019128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902255c7-a37c-4463-aecb-95985bdee890 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:40.052358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 504/1011] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:39 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:40.138205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19824a29-99fe-4e9b-ba5e-88c660decd32 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.138565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19824a29-99fe-4e9b-ba5e-88c660decd32 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.138755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19824a29-99fe-4e9b-ba5e-88c660decd32 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.140307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 508/1012] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:40.146159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02973e5d-2349-46fb-9754-2df7c5452242 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.146457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02973e5d-2349-46fb-9754-2df7c5452242 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.146741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02973e5d-2349-46fb-9754-2df7c5452242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.166305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-02973e5d-2349-46fb-9754-2df7c5452242 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:40.167554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-02973e5d-2349-46fb-9754-2df7c5452242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:40.244536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 505/1013] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:40.472497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c0fd-d117-4078-96a3-0709e790c988 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.473044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c0fd-d117-4078-96a3-0709e790c988 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.473044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51c0fd-d117-4078-96a3-0709e790c988 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.474514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 509/1014] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:40.480087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.480527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.480688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.499349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:40.500492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:40.515234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c0fd-d117-4078-96a3-0709e790c988 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:40.568024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 506/1015] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:40.618395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:40.624458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51c0fd-d117-4078-96a3-0709e790c988 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:40.658404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:40.659306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.659759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.660204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.661062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:40.662645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:40.672226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:40.673220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:46:40.674103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:40.675619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 510/1016] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:40.681427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:40.698231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:40.787641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ab62e-ebad-4c7c-8818-fb0ac3fe0416 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:40.822388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:40.823053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.823308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.823584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.824212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:40.824542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:40.825918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:40.830917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 507/1017] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:40.836548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:40.839017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:40.923187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:40.946387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3bead-f40e-4c3c-9b2c-2e8ab995065e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:40.952478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cef91-a26a-41de-8ead-4bf7fb79cba9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:40.986761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23bf71-2f3c-4ce9-9dfc-4c7a24736719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:40.987367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23bf71-2f3c-4ce9-9dfc-4c7a24736719 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.987645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23bf71-2f3c-4ce9-9dfc-4c7a24736719 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.987970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23bf71-2f3c-4ce9-9dfc-4c7a24736719 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.988561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:40.988675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23bf71-2f3c-4ce9-9dfc-4c7a24736719 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:40.989167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:40.989446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:40.989715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:40.990034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23bf71-2f3c-4ce9-9dfc-4c7a24736719 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:40.990330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:40.990631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:40.991969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:40.997973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 511/1018] 10.222.0.57 () {56 vars in 1112 bytes} [Thu Sep 15 12:46:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:41.004344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:41.012635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 508/1019] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:46:40 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:41.040766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:41.132441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc753268-4000-40da-b4f7-60c1f635cc66 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:41.179073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17fb4-cc56-4a2e-86f2-201b9c54b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:41.179690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17fb4-cc56-4a2e-86f2-201b9c54b8f7 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:41.179971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17fb4-cc56-4a2e-86f2-201b9c54b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:41.180243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17fb4-cc56-4a2e-86f2-201b9c54b8f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:41.181002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c17fb4-cc56-4a2e-86f2-201b9c54b8f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:41.182434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c17fb4-cc56-4a2e-86f2-201b9c54b8f7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:41.205472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 512/1020] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:42.296707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa7979-66f8-4ece-9c93-d95b74f27e15 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:42.297260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa7979-66f8-4ece-9c93-d95b74f27e15 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:42.297260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa7979-66f8-4ece-9c93-d95b74f27e15 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:42.298726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 509/1021] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:42.303786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:42.304174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:42.304376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:42.322529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:42.323811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:42.396951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 513/1022] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:42.667513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa7979-66f8-4ece-9c93-d95b74f27e15 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:42.773137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa7979-66f8-4ece-9c93-d95b74f27e15 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:42.808697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092e6c9-2010-4142-b93b-229e364001eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:42.809706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092e6c9-2010-4142-b93b-229e364001eb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:42.810190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092e6c9-2010-4142-b93b-229e364001eb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:42.810662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092e6c9-2010-4142-b93b-229e364001eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:42.811989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092e6c9-2010-4142-b93b-229e364001eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:42.813656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092e6c9-2010-4142-b93b-229e364001eb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:42.815735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 510/1023] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:42.821110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:42.829862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:42.925743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47869381-f165-4afc-9481-84f7fe8e18fe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:42.959695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddfcc6-a76c-4094-a250-131dbfd7a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:42.961644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddfcc6-a76c-4094-a250-131dbfd7a385 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:42.961644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddfcc6-a76c-4094-a250-131dbfd7a385 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:42.961644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddfcc6-a76c-4094-a250-131dbfd7a385 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:42.961644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddfcc6-a76c-4094-a250-131dbfd7a385 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:42.964471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddfcc6-a76c-4094-a250-131dbfd7a385 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:42.986674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 514/1024] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:46:42 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:43.248454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b08d9d-03f3-4dd5-8251-4c06a4668bd9 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:43.248823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b08d9d-03f3-4dd5-8251-4c06a4668bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:43.249018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b08d9d-03f3-4dd5-8251-4c06a4668bd9 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:43.250370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 511/1025] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:43.255785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b3e33-5b81-4126-8c26-ebc3ebbcbe9d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:43.256072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b3e33-5b81-4126-8c26-ebc3ebbcbe9d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:43.256374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b3e33-5b81-4126-8c26-ebc3ebbcbe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:43.276151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a45b3e33-5b81-4126-8c26-ebc3ebbcbe9d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:43.277514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a45b3e33-5b81-4126-8c26-ebc3ebbcbe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:43.337295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:43.351007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 515/1026] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:43.498621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f24336-8c1d-4d55-bac5-6cf19ee032dd None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:43.499108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f24336-8c1d-4d55-bac5-6cf19ee032dd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:43.499108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f24336-8c1d-4d55-bac5-6cf19ee032dd None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:43.500614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 512/1027] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:43.505669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:43.505966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:43.506250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:43.524845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:43.526203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:43.596189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 516/1028] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:43.625009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f24336-8c1d-4d55-bac5-6cf19ee032dd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:43.634252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:43.733510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f24336-8c1d-4d55-bac5-6cf19ee032dd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:43.767866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:43.768758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:43.769194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:43.769651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] PATH_INFO: `/v3/services/image` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:43.770367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:43.771902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:43.781931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:43.783097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:43.784228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:43.786644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 513/1029] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Sep 15 12:46:43 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:43.793034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:43.870883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:43.903261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1b530-f321-4c63-8cd4-aa6221c4f7d1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:43.928132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:43.943004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:43.943625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:43.943907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:43.944213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:43.944843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:43.945147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:43.946510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:43.948646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:43.956627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 517/1030] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:43.958000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:43.983356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5ead5-2046-486f-8a78-4a080eda5380 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.019069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.019599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.019876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.020146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.033055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:44.038184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:44.039131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:44.040067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 514/1031] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Sep 15 12:46:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:44.045942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.071926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7170bd-1401-49c1-80ff-24d1a865fda2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.116303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.116663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.116945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.117219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.117878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:44.118141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.119555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.124954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 518/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:44.130682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.152622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b7546-118b-4304-8f79-6444dee95299 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.183127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:44.185998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:44.191373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.192080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.192551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.192830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.193951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:44.194402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.196371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.211747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 515/1033] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:44.213251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.239795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88898f6a-08a3-41bf-b83e-3dafc96ece07 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.279275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.279804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.280091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.280411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.281149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] RBAC: Adding query filter params (interface=public, service_id=af3a2ab380a846aab0f76fd290619094, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:44.281462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.282826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.288045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 519/1034] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/endpoints?service_id=af3a2ab380a846aab0f76fd290619094&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:44.293828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.329657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700c27d-8f66-4d69-8253-cc7a58c3ff81 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.365445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.366067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.366355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.366609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.378398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.379839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:44.394766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:44.395919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:44.397016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:44.397016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:44.397016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:44.397016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 516/1035] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:44.402395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.403716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fbfb0-7f72-449d-a679-81485f11ee16 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.419635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:44.442245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.443172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.443675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.444278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.444972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.446443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.450720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:44.460348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 520/1036] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:46:44.513476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e987b7-fd58-4a73-847c-599204b863ff None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.549470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.550001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.550278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.550548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.551188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:44.551488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.552917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.570987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 517/1037] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:44.572605 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.684402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cecf1-f323-4612-ab0a-ace6e46c85c3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.719953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.720622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.720902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.721169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.728426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.730095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:44.741926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:44.742898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:44.743854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 521/1038] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:44.751047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.852607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e6013-e096-4fc6-91d9-a412173f392e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:44.855436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:44.889475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:44.890078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:44.890408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:44.890788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:44.891511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:44.891809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:44.893261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b738ee-480d-40c3-a264-cc7a19064df6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:44.915666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 518/1039] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:44.917610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:44.959513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:45.024634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a686d7-a65d-47b6-aaee-beb04181746c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.059435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.060047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.060354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.060631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.141910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:45.143277 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:45.183659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:45.219701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe98308-f8e0-4f44-bda9-2ee35c940802 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.220040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe98308-f8e0-4f44-bda9-2ee35c940802 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.220261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe98308-f8e0-4f44-bda9-2ee35c940802 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.221788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 519/1040] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:45.226678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.226965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.227245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.245538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:45.246737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:45.247235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 522/1041] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:46:45.252680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.321168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 520/1042] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:45.364651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffbaf3-ff97-4d6b-ba9d-dbd6e1f0c6ea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.399708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.400265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.400568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.401367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.407900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:45.413191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:45.414620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:45.429747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:45.430673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:45.431572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:45.431572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:45.431572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:45.431572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 523/1043] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:45.438748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.532408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:45.543896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28675a9d-65c5-4f6d-bb8b-3017652db1c4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.577305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.577912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.578193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.578464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.579089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:45.579390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:45.580784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:45.596303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 521/1044] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:45.597488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.598330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.701656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769ea69-ca92-4c46-a7c8-8ea15a438427 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.708436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369da55e-e384-412e-b9ae-f7a3f459991a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.736456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.737090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.737374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.737648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.744523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.744827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:45.745163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.745439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.745708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.745987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:45.756927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:45.757390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:45.758223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:45.759304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 522/1045] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:45.765281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:45.767621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:45.768587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:45.769430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:45.770523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 524/1046] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:45.778005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.781475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:45.836022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:45.873231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0ea5f-17ef-4cc7-ab43-952db4160bc8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.892807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c11f7-ec7a-4fd1-8c5f-547152d3149a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:45.910673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.911289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.911569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.911844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.912562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:45.912869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:45.914306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:45.928218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 523/1047] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:45.928533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:45.928954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:45.929231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:45.929504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:45.930151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:45.930454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:45.931805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:45.934171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:45.941711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 525/1048] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:45.947249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:46.008843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:46.041141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48577e4e-5f3a-4e42-a3e3-7cd3a5b622d0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:46.055053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046f1b6-26d2-4f50-8df0-867da074e364 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:46.070145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:46.080190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:46.080663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.080938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.081205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.081931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] RBAC: Adding query filter params (role.id=e0c7d64dc2c949c0972dcbc6b8712ed6, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=03f7ea8f3fb64e3d96fbe277e07b5435) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:46.082235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:46.083661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:46.089856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 524/1049] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/role_assignments?user.id=03f7ea8f3fb64e3d96fbe277e07b5435&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=e0c7d64dc2c949c0972dcbc6b8712ed6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:46.100051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:46.100788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.101074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.101345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] PATH_INFO: `/v3/users/217ffb7df3ab42e4bb13af6216e6769a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.117264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] RBAC: Authorizing `identity:get_user(user_id=217ffb7df3ab42e4bb13af6216e6769a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:46.118765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:46.124346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 526/1050] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:46:45 2022] GET /identity/v3/users/217ffb7df3ab42e4bb13af6216e6769a => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:46.127558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:46.242879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44e96-e757-417e-b9e1-da1dc1f76885 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:46.281497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:46.282471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.282953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.283443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.287337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:46.355182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:46.356881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:46.462855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 525/1051] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:46.464636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:46.569208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065151b2-9866-45a4-a714-1b1ae3976300 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:46.605539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f4cd4-32eb-4a21-80c5-52df457f6163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:46.606179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4cd4-32eb-4a21-80c5-52df457f6163 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.606461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4cd4-32eb-4a21-80c5-52df457f6163 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.606734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4cd4-32eb-4a21-80c5-52df457f6163 None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.607844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f4cd4-32eb-4a21-80c5-52df457f6163 None admin] RBAC: Authorizing `identity:get_project(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:46.609310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f4cd4-32eb-4a21-80c5-52df457f6163 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:46.616086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 527/1052] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:46:46 2022] GET /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:46.617084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:46.653244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:46.670243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8025e-67c3-4669-9a3a-c98f3bdc3e3c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.670524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8025e-67c3-4669-9a3a-c98f3bdc3e3c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.670799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8025e-67c3-4669-9a3a-c98f3bdc3e3c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.672170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 528/1053] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:46.677087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.677378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.677653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.696796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:46.698690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:46.728296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf078a-20d1-4f66-a8d0-30273061bd7c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:46.730320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:46.764098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:46.764741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:46.765021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:46.765298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:46.766028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a, user.id=217ffb7df3ab42e4bb13af6216e6769a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:46.766340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:46.767805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:46.773313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 529/1054] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:46:46.773451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 526/1055] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:46 2022] GET /identity/v3/role_assignments?user.id=217ffb7df3ab42e4bb13af6216e6769a&scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:47.045955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:47.155884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eee022-4b7d-4ef0-b47a-42f421a733eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:47.191344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:47.191976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:47.192254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:47.192552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:47.193119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:47.194530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:47.204241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:47.205244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:47.206082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 530/1056] 10.222.0.57 () {56 vars in 1105 bytes} [Thu Sep 15 12:46:47 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:47.212844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:47.315527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:47.319751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231a14-525d-495f-8cb5-ff8be43c98b0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:47.353130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:47.353730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:47.354582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:47.354854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:47.355470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] RBAC: Adding query filter params (name=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:47.355769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:47.357154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:47.366613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 527/1057] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:47.367864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:47.400418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:47.471149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2422d-a533-4af2-bc36-3cb66c5e18a7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:47.505089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:47.505700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:47.505984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:47.506264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:47.506882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:47.507189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:47.508587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06207107-ddad-4fff-889b-19cbc6806a31 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:47.518975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 531/1058] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:47.519930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:47.628134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b0430-ab05-4501-9d09-b8ae684a1a74 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:47.665529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a4bc4-6b3b-404c-b961-7d147e9db606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:47.666144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a4bc4-6b3b-404c-b961-7d147e9db606 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:47.666989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a4bc4-6b3b-404c-b961-7d147e9db606 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:47.667264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a4bc4-6b3b-404c-b961-7d147e9db606 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:47.667931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a4bc4-6b3b-404c-b961-7d147e9db606 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:47.669324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a4bc4-6b3b-404c-b961-7d147e9db606 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:47.694833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 528/1059] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:46:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:46:48.251408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54d841-238a-46dc-a106-5594321a27ca None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:48.251732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54d841-238a-46dc-a106-5594321a27ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:48.251970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54d841-238a-46dc-a106-5594321a27ca None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:48.253515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 532/1060] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:48.258432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:48.258727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:48.259006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:48.277728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:48.278911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:48.293364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:48.354736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 529/1061] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:48.621481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54d841-238a-46dc-a106-5594321a27ca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:48.718972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:48.726450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54d841-238a-46dc-a106-5594321a27ca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:48.759379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db8e13-3697-42a2-923f-276325c5b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:48.760566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db8e13-3697-42a2-923f-276325c5b67c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:48.760852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db8e13-3697-42a2-923f-276325c5b67c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:48.761129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db8e13-3697-42a2-923f-276325c5b67c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:48.772548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db8e13-3697-42a2-923f-276325c5b67c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-89db8e13-3697-42a2-923f-276325c5b67c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:48.778077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:48.779010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:48.779870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:48.781161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 533/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Sep 15 12:46:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:48.785233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:48.888812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7ad7d-32ac-4fb1-9105-3883534e6508 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:48.916781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e37062-8daa-4db3-ba05-ddca7c5dcb8e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:48.917131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e37062-8daa-4db3-ba05-ddca7c5dcb8e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:48.917460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e37062-8daa-4db3-ba05-ddca7c5dcb8e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:48.919032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 534/1063] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:46:48.923873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:48.924160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:48.924471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:48.929522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:48.930139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:48.930416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:48.930953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:48.931618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:48.931960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:48.933717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:48.942188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 530/1064] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:46:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:48.944303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:48.945514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:48.948507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:48.960695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.023994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 535/1065] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:49.059536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f7631-5296-478f-a5a5-71d7e23056e5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.099543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.100182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.100492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.100768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.111699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.113164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:49.128082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:49.129162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:49.130033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:49.130033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:49.130033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:49.130789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 531/1066] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:49.134884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.221357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.237942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec45538-a22a-4eb8-9fe1-8ebb76bf3569 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.274210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.274826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.275102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.275368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.276029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:49.276503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.277868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.293488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 536/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:49.294893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.295219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.357997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.405188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cd351-bbb8-4392-9a1f-b5b2c401caaf None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.409315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03014f00-90a1-40f0-9d10-5bfd0abdb8b5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.445165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.445799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.446089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.446250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.446388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.446852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.447136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.447457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.456451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.457511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:49.458210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:49.462475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:49.463639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:49.464811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 532/1068] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:49.469578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:49.471445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:49.472375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:49.473322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:49.477183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 537/1069] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:49.478361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.488155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.577946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c220fc-b8e1-4fb9-adf4-3a3c72f041d6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.590535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5b255-60d2-44ba-855d-f9baaf6271d9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.597790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.614534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.615207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.615494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.615770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.616475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:49.616789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.618165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.628201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 533/1070] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:49.633547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.634017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.634263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.634545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.634816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.636070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:49.636405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.638897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.656872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 538/1071] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:49.662775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.729868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.743654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe97bca-ce60-4108-b510-c146c8a4ecf0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.774176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71823c07-7337-425c-b85c-fba6c9efbca7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.781268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.781922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.782200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.782474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] PATH_INFO: `/v3/users/217ffb7df3ab42e4bb13af6216e6769a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.783672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] RBAC: Authorizing `identity:get_user(user_id=217ffb7df3ab42e4bb13af6216e6769a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.785173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.786824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 534/1072] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/users/217ffb7df3ab42e4bb13af6216e6769a => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:49.793724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.823013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.823638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.824037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.824337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.835174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.836767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.843318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:49.855929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:49.857288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:49.858186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:49.858186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:49.858186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:49.858186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 539/1073] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:49.863132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.898431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719c138-fa43-4c55-abb9-868219536908 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.931594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51393d94-61f9-4973-bb3a-c5513a7e96d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:49.933284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51393d94-61f9-4973-bb3a-c5513a7e96d4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.933566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51393d94-61f9-4973-bb3a-c5513a7e96d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.933834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51393d94-61f9-4973-bb3a-c5513a7e96d4 None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.934944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51393d94-61f9-4973-bb3a-c5513a7e96d4 None admin] RBAC: Authorizing `identity:get_project(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:49.937073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51393d94-61f9-4973-bb3a-c5513a7e96d4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:49.938194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 535/1074] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:49.939749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db43c16-7f69-43f7-a909-b85a30a54033 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:49.940027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db43c16-7f69-43f7-a909-b85a30a54033 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:49.940323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db43c16-7f69-43f7-a909-b85a30a54033 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:49.941661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 536/1075] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:49.943913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db43c16-7f69-43f7-a909-b85a30a54033 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:49.969261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c7801-6257-47dc-8659-22ee83e8d7f8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:49.985090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:50.003859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.004490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.004793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.005063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.005689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:50.005987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.007348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac353f6-e7b5-46f3-b0f2-ceecd8f130ea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.020135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 540/1076] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:50.021352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.021638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.021907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.040170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:50.042451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:50.051187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db43c16-7f69-43f7-a909-b85a30a54033 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:50.068563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:50.087349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.088566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.088846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.089123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a/users/217ffb7df3ab42e4bb13af6216e6769a/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.091168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] RBAC: Authorizing `identity:create_grant(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a, user_id=217ffb7df3ab42e4bb13af6216e6769a, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.092634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.108680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 537/1077] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:46:49 2022] PUT /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a/users/217ffb7df3ab42e4bb13af6216e6769a/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:46:50.118274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:50.120256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 541/1078] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:50.232160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab0232-577f-4541-9fc2-3254a952d1bf None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:50.269708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.270261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.270588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.270863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.279742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.281389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:50.293019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:50.293944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:50.295319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 538/1079] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:50.301350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:50.394022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:50.411953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d4c47-9e8d-4be2-b991-abc8493169ef None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:50.420388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:50.439865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:50.452372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.452991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.453281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.453561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.454248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:50.454561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.455999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.473775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 542/1080] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:50.474802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:50.512998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ce6e-9fcb-4bf0-afc1-532d1f5b473c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:50.549262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.550119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.550548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.550975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.577960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f60ef-fa72-40eb-ac36-09f8ec24b260 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:50.616752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.617449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.617698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.617968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/03f7ea8f3fb64e3d96fbe277e07b5435/roles/e0c7d64dc2c949c0972dcbc6b8712ed6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.620351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=03f7ea8f3fb64e3d96fbe277e07b5435, role_id=e0c7d64dc2c949c0972dcbc6b8712ed6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.621769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.630959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.632703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.636858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 543/1081] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:46:50 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/03f7ea8f3fb64e3d96fbe277e07b5435/roles/e0c7d64dc2c949c0972dcbc6b8712ed6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:46:50.677875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:50.746754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 539/1082] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:50.752507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:50.784262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:50.870138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2476c2-f6eb-4459-a066-3fa427ac7b00 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:50.903988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:50.904884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:50.905333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:50.905760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:50.917142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:50.918882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a08657c5-cd48-4aa9-97de-56f322fcad95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:50.934715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:50.935872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:50.936881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:50.936881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:50.936881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:50.937717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 544/1083] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:50.943759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:51.049661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac674b-acec-4058-82e6-74932e9a13bc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:51.083431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:51.084041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:51.088508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:51.088508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:51.088508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:51.088508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:51.088508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443154e-2f68-4fe5-8827-409108a452ed None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:51.098144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:51.108305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 540/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:52.304295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6223-b012-4f04-8bff-4fa7585aa08e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:52.304878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6223-b012-4f04-8bff-4fa7585aa08e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:52.304878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6223-b012-4f04-8bff-4fa7585aa08e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:52.306218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 545/1085] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:52.312406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19477e-8645-46a1-a2fb-58737f5fe328 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:52.312872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19477e-8645-46a1-a2fb-58737f5fe328 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:52.312980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19477e-8645-46a1-a2fb-58737f5fe328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:52.331658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ef19477e-8645-46a1-a2fb-58737f5fe328 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:52.332876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ef19477e-8645-46a1-a2fb-58737f5fe328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:52.398261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 541/1086] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:52.671281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6223-b012-4f04-8bff-4fa7585aa08e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:52.755874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:52.779372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6223-b012-4f04-8bff-4fa7585aa08e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:52.813005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:52.813632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:52.813913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:52.814187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:52.824903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:52.826907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5570f-ba51-4698-a7eb-9afa1bc0c9e4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:52.827192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5570f-ba51-4698-a7eb-9afa1bc0c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:52.827463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5570f-ba51-4698-a7eb-9afa1bc0c9e4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:52.828958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 542/1087] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:52.829971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:52.831106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:52.832021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 546/1088] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:46:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:52.834256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:52.834538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:52.834810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:52.836943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:52.852937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:52.855108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:52.921871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 543/1089] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:52.937883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab218e2-aba6-49da-8971-6cfe5d413ff8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:52.970861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:52.971465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:52.971749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:52.972032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:52.972978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:52.973292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:52.974654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:52.980182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:52.986981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 547/1090] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:46:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:52.990081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.048149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.090532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c587aa-6963-4828-bd26-0a141e186d4c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.125277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.125906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.126187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.126457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] PATH_INFO: `/v3/users/217ffb7df3ab42e4bb13af6216e6769a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.127578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] RBAC: Authorizing `identity:get_user(user_id=217ffb7df3ab42e4bb13af6216e6769a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.129034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.130592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 544/1091] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:46:52 2022] GET /identity/v3/users/217ffb7df3ab42e4bb13af6216e6769a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:53.136842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.191610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.236394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff19e5-3a3d-4065-8334-fde24fede7d3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.270959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.271563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.271840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.272120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.290704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e791f9-4409-49f9-949e-767b07679a60 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.315817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.325812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.326464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.326738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.327009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.336011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.337195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:46:53.339419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.341239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:46:53.342763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:53.343755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:53.344639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 545/1092] 10.222.0.57 () {56 vars in 1120 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:53.349876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.441041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 548/1093] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:46:53.446993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.447208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa276b-a877-4bbb-b04f-3710994d28ef None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.481754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.482352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.482630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.483542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.483542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:53.483840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.485252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.492954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 546/1094] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:53.499385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.545033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.547568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.547972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a961d-99d0-4857-8cee-f3737700749f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.584688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.585321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.585602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.585875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.586985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] RBAC: Authorizing `identity:get_project(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.588530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.590179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 549/1095] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:53.596253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.597769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af187c-a6eb-4291-8974-f5bc29af2f15 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.630618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.631231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.631508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.631777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.642812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.644238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:53.657836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:53.658933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:53.659833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:53.659833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:53.659833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:53.659833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 547/1096] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:53.664733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.696388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f1162-c252-41ee-9296-c7262cbac66e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.730928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.731538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.731816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.732082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.732829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a, user.id=217ffb7df3ab42e4bb13af6216e6769a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:53.733134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.734529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.740115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 550/1097] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/role_assignments?user.id=217ffb7df3ab42e4bb13af6216e6769a&scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:53.769595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.773072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf14bfc-b0e0-41b1-884e-ecc41bab2359 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.809790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.810397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.810676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.810947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.811618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:53.811919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.813358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.827012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 548/1098] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:46:53.832948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:53.866017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:53.939765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f92350-8cb2-4f65-a19c-cf7812e9a8fe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:53.973538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:53.974174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:53.974456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:53.974726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:53.983511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:53.984907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:46:53.996597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:53.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:46:53.998752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 551/1099] 10.222.0.57 () {56 vars in 1093 bytes} [Thu Sep 15 12:46:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:54.003639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:54.104074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ca550-c843-4393-ab1c-2f60b761fd36 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:54.139103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:54.139708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:54.139990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:54.140265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:54.141039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:54.141346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:54.142791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:54.160587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 549/1100] 10.222.0.57 () {56 vars in 1133 bytes} [Thu Sep 15 12:46:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:54.167252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:54.258270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:54.267703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d181c-6c24-4a05-9871-c3c68127501a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:54.302079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:54.302688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:54.302969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:54.303241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:54.369740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:54.371189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:54.458172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:54.468576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 552/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:54.474551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:54.493444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:54.573970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62cad9-1180-4c61-b8da-160a5d8c6e63 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:54.608969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:54.609591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:54.609868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:54.610138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:54.619833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:54.621290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:46:54.635870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:54.637411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:46:54.638325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:46:54.638325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:46:54.638325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:54.638325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 550/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:46:54.643184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:54.741570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2cf30-d862-411c-8556-572b044d7f91 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:54.775146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:54.776625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:54.776953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:54.777227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:54.777854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:54.778155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:54.779508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:54.790270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 553/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:54.796392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:54.863695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:54.894926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfa293-ae7a-46b6-97d4-7ac2c921da97 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:54.927761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:54.928421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:54.928713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:54.928970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:54.935477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:54.936962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:46:54.948489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:46:54.949458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:46:54.950756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 551/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:46:54.955440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:55.007985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:55.055286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb65cb7-7503-4a3d-a8aa-8d604761948a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:55.090819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:55.091432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:55.091722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:55.092707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:55.092707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:55.094809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:55.094809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c9963-b7ac-4ca5-b834-1e152d8b21f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:55.107747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 554/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:55.113621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:55.212929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154e717-956c-4bfa-a1b4-5162c5d80271 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:55.239691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:55.246515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:55.247211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:55.247500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:55.247775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:55.248545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] RBAC: Adding query filter params (role.id=e0c7d64dc2c949c0972dcbc6b8712ed6, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=03f7ea8f3fb64e3d96fbe277e07b5435) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:46:55.248858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:55.250344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebf8d8-e3db-456b-8531-77efd2e749b4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:55.256118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 552/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:55 2022] GET /identity/v3/role_assignments?user.id=03f7ea8f3fb64e3d96fbe277e07b5435&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=e0c7d64dc2c949c0972dcbc6b8712ed6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:55.935184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4924-d32f-49ee-a916-e281cab95fec None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:55.935690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4924-d32f-49ee-a916-e281cab95fec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:55.935769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4924-d32f-49ee-a916-e281cab95fec None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:55.937152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 555/1107] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:55.942595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:55.943013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:55.943151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:55.961427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:55.962621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:56.033910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 553/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:56.300393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4924-d32f-49ee-a916-e281cab95fec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:56.399671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4924-d32f-49ee-a916-e281cab95fec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:56.412988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:56.433469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec956dd1-86aa-4aeb-9173-0938b53e0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:56.434091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec956dd1-86aa-4aeb-9173-0938b53e0339 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:56.434374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec956dd1-86aa-4aeb-9173-0938b53e0339 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:56.434651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec956dd1-86aa-4aeb-9173-0938b53e0339 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:56.435753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec956dd1-86aa-4aeb-9173-0938b53e0339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:56.437210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec956dd1-86aa-4aeb-9173-0938b53e0339 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:56.438758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 556/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:46:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:56.444444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:56.541037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5333c3-c226-4d9a-b793-117b781c2d2c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:56.563968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:56.575186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcec2af-7197-4f6a-b237-c83af85ebbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:56.575779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcec2af-7197-4f6a-b237-c83af85ebbe0 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:56.576056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcec2af-7197-4f6a-b237-c83af85ebbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:56.576359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcec2af-7197-4f6a-b237-c83af85ebbe0 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:56.577014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcec2af-7197-4f6a-b237-c83af85ebbe0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:56.578431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcec2af-7197-4f6a-b237-c83af85ebbe0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:56.599560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 554/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:46:56 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:46:58.793975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15185e-b811-4b35-8094-91bd76d32676 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:58.794505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15185e-b811-4b35-8094-91bd76d32676 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:58.794505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15185e-b811-4b35-8094-91bd76d32676 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:58.795838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 557/1111] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:46:58.801328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:58.801739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:58.801881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:58.820526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:46:58.821704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:46:58.886155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 555/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:46:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:46:59.153888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15185e-b811-4b35-8094-91bd76d32676 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:59.253409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15185e-b811-4b35-8094-91bd76d32676 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:59.286178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:59.286792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:59.287073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:59.287343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] PATH_INFO: `/v3/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:59.288745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] RBAC: Authorizing `identity:get_role(role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:59.290096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:59.291905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 558/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:46:59 2022] GET /identity/v3/roles/0f55ac654fb247249b38e3c701826f02 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:59.297713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:59.393454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194eb32-8178-46ad-a1b5-019bc8705246 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:59.403816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:59.427837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:59.428491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:59.428772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:59.429045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] PATH_INFO: `/v3/users/754aa9d6dee049878f03814362635724` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:59.442063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] RBAC: Authorizing `identity:get_user(user_id=754aa9d6dee049878f03814362635724)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:59.443505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:59.445108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 556/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:46:59 2022] GET /identity/v3/users/754aa9d6dee049878f03814362635724 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:59.451161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:59.499155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:59.548582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5d1d1-ca43-46ff-88db-913773cbf1c6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:59.580675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:59.581267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:59.581549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:59.581821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:59.646933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:59.648317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:59.693748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:46:59.744554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 559/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:46:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:46:59.750563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:59.845566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0716-e57e-4584-9c51-986bae41b073 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:46:59.877732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8179ef-b6d9-405c-bf36-0c6dfb1f887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:46:59.878342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8179ef-b6d9-405c-bf36-0c6dfb1f887f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:46:59.878617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8179ef-b6d9-405c-bf36-0c6dfb1f887f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:46:59.878885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8179ef-b6d9-405c-bf36-0c6dfb1f887f None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:46:59.879988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8179ef-b6d9-405c-bf36-0c6dfb1f887f None admin] RBAC: Authorizing `identity:get_project(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:46:59.881420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8179ef-b6d9-405c-bf36-0c6dfb1f887f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:46:59.882996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 557/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:46:59 2022] GET /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:46:59.890140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:46:59.988508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4394581c-9f44-40d4-b981-a3aa08884ddb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:00.021460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:00.022051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:00.022331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:00.022601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:00.023312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] RBAC: Adding query filter params (role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a, user.id=754aa9d6dee049878f03814362635724) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:00.023618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:00.025058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c957-1088-4753-bd2a-032e2c35b544 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:00.030985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 560/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:46:59 2022] GET /identity/v3/role_assignments?user.id=754aa9d6dee049878f03814362635724&scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a&role.id=0f55ac654fb247249b38e3c701826f02 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:02.151109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8857fec-2d9a-4fb9-aa3e-e4b6317e44ed None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:02.151799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8857fec-2d9a-4fb9-aa3e-e4b6317e44ed None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:02.151894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8857fec-2d9a-4fb9-aa3e-e4b6317e44ed None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:02.153215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 558/1118] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:02.158435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:02.158830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:02.158995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:02.176760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:02.177953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:02.186112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:02.245548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 561/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:02.510701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8857fec-2d9a-4fb9-aa3e-e4b6317e44ed None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:02.545794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:02.609236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8857fec-2d9a-4fb9-aa3e-e4b6317e44ed None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:02.643003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:02.643618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:02.643896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:02.644168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] PATH_INFO: `/v3/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:02.645560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] RBAC: Authorizing `identity:get_role(role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:02.646924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:02.648223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 559/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:02 2022] GET /identity/v3/roles/0f55ac654fb247249b38e3c701826f02 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:02.654150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:02.749193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667792f9-ca87-4ef4-a839-5d83feb3d18e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:02.782092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:02.782650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:02.782928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:02.783197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] PATH_INFO: `/v3/users/754aa9d6dee049878f03814362635724` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:02.784312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] RBAC: Authorizing `identity:get_user(user_id=754aa9d6dee049878f03814362635724)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:02.785718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:02.787248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 562/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:02 2022] GET /identity/v3/users/754aa9d6dee049878f03814362635724 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:02.792853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:02.891072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1ebd2-4a9d-4fe4-a043-8f35a50956fe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:02.893343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:02.924451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ade7-5e59-423b-a0d1-98cf1f1fedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:02.925070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ade7-5e59-423b-a0d1-98cf1f1fedd2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:02.925351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ade7-5e59-423b-a0d1-98cf1f1fedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:02.925624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ade7-5e59-423b-a0d1-98cf1f1fedd2 None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:02.926729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ade7-5e59-423b-a0d1-98cf1f1fedd2 None admin] RBAC: Authorizing `identity:get_project(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:02.928152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ade7-5e59-423b-a0d1-98cf1f1fedd2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:02.929777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 560/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:02 2022] GET /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:02.936346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:02.948328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:03.033366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c20b2-29a4-472d-bef5-d72bda93ab7d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:03.066734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061b60-5747-4eea-b0ff-58898ea0b919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:03.067379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061b60-5747-4eea-b0ff-58898ea0b919 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:03.067661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061b60-5747-4eea-b0ff-58898ea0b919 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:03.067935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061b60-5747-4eea-b0ff-58898ea0b919 None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a/users/754aa9d6dee049878f03814362635724/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:03.070081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e061b60-5747-4eea-b0ff-58898ea0b919 None admin] RBAC: Authorizing `identity:create_grant(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a, user_id=754aa9d6dee049878f03814362635724, role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:03.071494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e061b60-5747-4eea-b0ff-58898ea0b919 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:03.084322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 563/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:47:02 2022] PUT /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a/users/754aa9d6dee049878f03814362635724/roles/0f55ac654fb247249b38e3c701826f02 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:47:05.225596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e75c9-395f-42c9-ac4a-19cd7ab57335 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:05.226117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e75c9-395f-42c9-ac4a-19cd7ab57335 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:05.226117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e75c9-395f-42c9-ac4a-19cd7ab57335 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:05.227455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 561/1124] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:05.232911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:05.233356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:05.233450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:05.251053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:05.252218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:05.316696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:05.321715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 564/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:05.591777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e75c9-395f-42c9-ac4a-19cd7ab57335 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:05.690044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e75c9-395f-42c9-ac4a-19cd7ab57335 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:05.722772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:05.723388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:05.723681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:05.723937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] PATH_INFO: `/v3/roles/0f55ac654fb247249b38e3c701826f02` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:05.725333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] RBAC: Authorizing `identity:get_role(role_id=0f55ac654fb247249b38e3c701826f02)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:05.726692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:05.727969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 562/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:05 2022] GET /identity/v3/roles/0f55ac654fb247249b38e3c701826f02 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:05.734072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:05.828902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef772aa5-6fe6-4ff6-b9bc-49c6e124971c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:05.861440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:05.861995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:05.862273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:05.862541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] PATH_INFO: `/v3/users/754aa9d6dee049878f03814362635724` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:05.863635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] RBAC: Authorizing `identity:get_user(user_id=754aa9d6dee049878f03814362635724)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:05.865062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:05.866596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 565/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:05 2022] GET /identity/v3/users/754aa9d6dee049878f03814362635724 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:05.872805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:05.912978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:05.969618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545b586-bf35-4484-8b28-890aabdd4a8d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:06.002660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:06.003251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:06.003530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:06.003803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:06.068500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:06.069935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:06.109085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:06.166216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 563/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:47:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:47:06.172177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:06.239093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:06.271852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b129e-d76d-4e8a-8fe2-b23d670b8d73 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:06.305600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e878-0936-4e82-a163-163c57eb56f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:06.306220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e878-0936-4e82-a163-163c57eb56f6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:06.306611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e878-0936-4e82-a163-163c57eb56f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:06.307060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e878-0936-4e82-a163-163c57eb56f6 None admin] PATH_INFO: `/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:06.308158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e878-0936-4e82-a163-163c57eb56f6 None admin] RBAC: Authorizing `identity:get_project(project_id=b534c256a8a8429dbdb43ee3d9ab2c9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:06.309604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e878-0936-4e82-a163-163c57eb56f6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:06.311200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 566/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:06 2022] GET /identity/v3/projects/b534c256a8a8429dbdb43ee3d9ab2c9a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:06.317458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:06.412586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce8304-3352-4247-99b8-bd154875f674 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:06.445278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:06.445805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:06.446079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:06.446347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:06.447076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] RBAC: Adding query filter params (role.id=0f55ac654fb247249b38e3c701826f02, scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a, user.id=754aa9d6dee049878f03814362635724) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:06.447385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:06.448837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a2271-d6fe-4310-a485-313dc212267a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:06.454337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 564/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:47:06 2022] GET /identity/v3/role_assignments?user.id=754aa9d6dee049878f03814362635724&scope.project.id=b534c256a8a8429dbdb43ee3d9ab2c9a&role.id=0f55ac654fb247249b38e3c701826f02 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:08.592756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95c85-0a77-4b03-a3af-8bbe63fa7dab None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:08.593287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95c85-0a77-4b03-a3af-8bbe63fa7dab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:08.593287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95c85-0a77-4b03-a3af-8bbe63fa7dab None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:08.594636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 567/1131] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:08.599747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:08.600161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:08.600312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:08.608438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:08.620266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:08.621470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:08.687104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 565/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:08.955409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc95c85-0a77-4b03-a3af-8bbe63fa7dab None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:09.054666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc95c85-0a77-4b03-a3af-8bbe63fa7dab None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:09.088520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02019235-be20-4e9a-a0af-5b5df2d75d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:09.089139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02019235-be20-4e9a-a0af-5b5df2d75d8a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:09.089420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02019235-be20-4e9a-a0af-5b5df2d75d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:09.089692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02019235-be20-4e9a-a0af-5b5df2d75d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:09.090807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02019235-be20-4e9a-a0af-5b5df2d75d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:09.092238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02019235-be20-4e9a-a0af-5b5df2d75d8a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:09.093833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 568/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:47:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:09.099558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:09.194987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8b31b-a675-4fcb-8f67-51374bad673c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:09.227784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195334e-8a1c-4177-96d4-142b2260c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:09.228465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195334e-8a1c-4177-96d4-142b2260c159 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:09.228747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195334e-8a1c-4177-96d4-142b2260c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:09.229018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195334e-8a1c-4177-96d4-142b2260c159 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:09.229654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195334e-8a1c-4177-96d4-142b2260c159 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:09.231070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195334e-8a1c-4177-96d4-142b2260c159 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:09.235425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:09.258838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 566/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:47:09 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:47:11.438487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213388-1374-45f1-8815-0b5c613c6f50 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:11.438985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213388-1374-45f1-8815-0b5c613c6f50 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:11.438985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213388-1374-45f1-8815-0b5c613c6f50 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:11.440337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 569/1135] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:11.445606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:11.446017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:11.446152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:11.465159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:11.466326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:11.530449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 567/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:11.802566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213388-1374-45f1-8815-0b5c613c6f50 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:11.807298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:11.902124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213388-1374-45f1-8815-0b5c613c6f50 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:11.935574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3297c5a-e368-4b25-81ba-b8a8f29a9821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:11.936196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3297c5a-e368-4b25-81ba-b8a8f29a9821 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:11.936502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3297c5a-e368-4b25-81ba-b8a8f29a9821 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:11.936774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3297c5a-e368-4b25-81ba-b8a8f29a9821 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:11.937873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3297c5a-e368-4b25-81ba-b8a8f29a9821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:11.939297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3297c5a-e368-4b25-81ba-b8a8f29a9821 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:11.940881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 570/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:47:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:11.946551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:12.042650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0355d-a66e-4b70-bb03-6da7c76f431f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:12.058917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:12.082705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d3792-3d6c-412d-ac58-869655ac13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:12.083310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d3792-3d6c-412d-ac58-869655ac13bb None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:12.083590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d3792-3d6c-412d-ac58-869655ac13bb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:12.083860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d3792-3d6c-412d-ac58-869655ac13bb None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:12.084549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d3792-3d6c-412d-ac58-869655ac13bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:12.085981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d3792-3d6c-412d-ac58-869655ac13bb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:12.107559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 568/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:47:11 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:47:14.280965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322e08d-8250-4d39-9b31-49f61d3ea2fe None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:14.281500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322e08d-8250-4d39-9b31-49f61d3ea2fe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:14.281500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322e08d-8250-4d39-9b31-49f61d3ea2fe None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:14.282835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 571/1139] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:14.288075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:14.288473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:14.288662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:14.307559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:14.308775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:14.373144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 569/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:14.647378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322e08d-8250-4d39-9b31-49f61d3ea2fe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:14.718437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:14.748035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322e08d-8250-4d39-9b31-49f61d3ea2fe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:14.781015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:14.781634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:14.781916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:14.782189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:14.792565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:47:14.797357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:14.798399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:47:14.799558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 572/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:47:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:14.804460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:14.901518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f57216-fe26-4538-8e92-d8b743a14965 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:14.902954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:14.935759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:14.936383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:14.936665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:14.936939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:14.937596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:14.937900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:14.939268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:14.947463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 570/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:14.953630 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:15.051393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d8adf-ecac-41b6-9f66-91bb30273a00 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:15.079508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:15.084955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:15.085586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:15.085861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:15.086129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] PATH_INFO: `/v3/users/9228079f7603433fbfa02953683458cc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:15.100319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] RBAC: Authorizing `identity:get_user(user_id=9228079f7603433fbfa02953683458cc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:15.101753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:15.103423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 573/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:14 2022] GET /identity/v3/users/9228079f7603433fbfa02953683458cc => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:15.109303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:15.206147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd9b56-bb80-4436-8723-cf2cb575a4ca None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:15.238565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:15.239152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:15.239429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:15.239698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:15.269636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:15.304947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:15.306887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:15.401713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 571/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:47:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:47:15.407598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:15.504308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883b888-8d9c-49f7-9cb2-bdcc68f6f499 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:15.537510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968358d-e877-44c3-96e9-95c407813415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:15.538124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968358d-e877-44c3-96e9-95c407813415 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:15.538407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968358d-e877-44c3-96e9-95c407813415 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:15.538683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968358d-e877-44c3-96e9-95c407813415 None admin] PATH_INFO: `/v3/projects/9966aa0955b949f3bffc170f9bc010a8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:15.539788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968358d-e877-44c3-96e9-95c407813415 None admin] RBAC: Authorizing `identity:get_project(project_id=9966aa0955b949f3bffc170f9bc010a8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:15.541270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968358d-e877-44c3-96e9-95c407813415 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:15.542866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 574/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:15 2022] GET /identity/v3/projects/9966aa0955b949f3bffc170f9bc010a8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:15.548811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:15.609632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:15.646611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e4265-5e96-47b9-b570-adb79347cc6d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:15.679460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:15.680047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:15.680352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:15.680626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:15.681906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=9966aa0955b949f3bffc170f9bc010a8, user.id=9228079f7603433fbfa02953683458cc) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:15.682216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:15.683623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeeda38-8c5d-4242-aee6-3e88ef5c6b44 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:15.688856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 572/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:47:15 2022] GET /identity/v3/role_assignments?user.id=9228079f7603433fbfa02953683458cc&scope.project.id=9966aa0955b949f3bffc170f9bc010a8&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:17.806652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d609b81-7f7a-4960-8fc5-fd154fc64062 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:17.807184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d609b81-7f7a-4960-8fc5-fd154fc64062 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:17.807184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d609b81-7f7a-4960-8fc5-fd154fc64062 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:17.808562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 575/1147] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:17.813640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:17.814018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:17.814188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:17.832927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:17.834107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:17.898187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 573/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:18.173915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d609b81-7f7a-4960-8fc5-fd154fc64062 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:18.232471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:18.274989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d609b81-7f7a-4960-8fc5-fd154fc64062 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:18.308715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:18.309329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:18.309608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:18.309877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:18.319619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:47:18.325185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:18.326289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:47:18.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 576/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:18.332528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:18.389283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:18.429255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59defa-00cd-4048-9991-f531c084bf68 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:18.463584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:18.464180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:18.464482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:18.464748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:18.465395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:18.465692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:18.467043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:18.474750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 574/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:18.481350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:18.578201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc38010-4a2e-4403-9274-ef5dadbe5240 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:18.595342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:18.612372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:18.612994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:18.613282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:18.613558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] PATH_INFO: `/v3/users/9228079f7603433fbfa02953683458cc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:18.614670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] RBAC: Authorizing `identity:get_user(user_id=9228079f7603433fbfa02953683458cc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:18.616090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:18.617688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 577/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:18 2022] GET /identity/v3/users/9228079f7603433fbfa02953683458cc => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:18.623391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:18.720059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c761a-6df1-46fd-b80c-bb5f75645cde None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:18.738594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:18.752826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafb9a2-cab4-4320-8f35-e02cf2bd9f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:18.753443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafb9a2-cab4-4320-8f35-e02cf2bd9f08 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:18.753725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafb9a2-cab4-4320-8f35-e02cf2bd9f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:18.753998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafb9a2-cab4-4320-8f35-e02cf2bd9f08 None admin] PATH_INFO: `/v3/projects/9966aa0955b949f3bffc170f9bc010a8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:18.755103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafb9a2-cab4-4320-8f35-e02cf2bd9f08 None admin] RBAC: Authorizing `identity:get_project(project_id=9966aa0955b949f3bffc170f9bc010a8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:18.756560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafb9a2-cab4-4320-8f35-e02cf2bd9f08 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:18.758152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 575/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:18 2022] GET /identity/v3/projects/9966aa0955b949f3bffc170f9bc010a8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:18.764888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:18.862684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490f298-cb35-4730-a8dc-d3c3ce4b1d88 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:18.896086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327736a2-6473-4fe0-a554-4cea1ddd726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:18.896767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327736a2-6473-4fe0-a554-4cea1ddd726a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:18.897051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327736a2-6473-4fe0-a554-4cea1ddd726a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:18.897327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327736a2-6473-4fe0-a554-4cea1ddd726a None admin] PATH_INFO: `/v3/projects/9966aa0955b949f3bffc170f9bc010a8/users/9228079f7603433fbfa02953683458cc/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:18.899215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327736a2-6473-4fe0-a554-4cea1ddd726a None admin] RBAC: Authorizing `identity:create_grant(project_id=9966aa0955b949f3bffc170f9bc010a8, user_id=9228079f7603433fbfa02953683458cc, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:18.901291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327736a2-6473-4fe0-a554-4cea1ddd726a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:18.913995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 578/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:47:18 2022] PUT /identity/v3/projects/9966aa0955b949f3bffc170f9bc010a8/users/9228079f7603433fbfa02953683458cc/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:47:21.028771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ae8d0-8535-4e42-b0b5-4f55eadf50f3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:21.029299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ae8d0-8535-4e42-b0b5-4f55eadf50f3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:21.029299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ae8d0-8535-4e42-b0b5-4f55eadf50f3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:21.030644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 576/1154] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:21.035862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:21.036263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:21.036437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:21.055249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:21.057068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:21.098306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:21.124733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 579/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:21.397156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ae8d0-8535-4e42-b0b5-4f55eadf50f3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:21.496732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ae8d0-8535-4e42-b0b5-4f55eadf50f3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:21.529408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:21.530023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:21.530305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:21.530580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:21.541327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:47:21.546112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:21.547191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:47:21.548096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 577/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:21.553128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:21.649441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91940f-407c-4b5d-b919-ac1c8bc75c33 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:21.682396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:21.682991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:21.683274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:21.683545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:21.684196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:21.684525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:21.685879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:21.693651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 580/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:21.700400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:21.748747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:21.798829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c583-864b-4917-a280-9b3fa1d18271 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:21.830966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:21.831583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:21.831863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:21.832137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] PATH_INFO: `/v3/users/9228079f7603433fbfa02953683458cc` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:21.833266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] RBAC: Authorizing `identity:get_user(user_id=9228079f7603433fbfa02953683458cc)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:21.834697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:21.836233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 578/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:21 2022] GET /identity/v3/users/9228079f7603433fbfa02953683458cc => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:21.842493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:21.882439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:21.942087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d780-c08b-4421-acf3-e25da3c11ca1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:21.975766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:21.976394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:21.976679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:21.976955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:22.042410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:22.043844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:22.081082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:22.141641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 581/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:47:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:47:22.147532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:22.243087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba45c6-2b89-4104-8e1c-f200eb1f180c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:22.258749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:22.277320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac548c9-1237-46e8-b60e-24a34e324ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:22.277934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac548c9-1237-46e8-b60e-24a34e324ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:22.278207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac548c9-1237-46e8-b60e-24a34e324ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:22.278472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac548c9-1237-46e8-b60e-24a34e324ef6 None admin] PATH_INFO: `/v3/projects/9966aa0955b949f3bffc170f9bc010a8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:22.279558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac548c9-1237-46e8-b60e-24a34e324ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=9966aa0955b949f3bffc170f9bc010a8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:22.286182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac548c9-1237-46e8-b60e-24a34e324ef6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:22.286182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 579/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:22 2022] GET /identity/v3/projects/9966aa0955b949f3bffc170f9bc010a8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:22.288951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:22.385182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb40e36-7ec0-43e5-9f04-e95d1124911c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:22.418139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:22.418725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:22.419005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:22.419279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:22.419985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=9966aa0955b949f3bffc170f9bc010a8, user.id=9228079f7603433fbfa02953683458cc) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:22.420316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:22.421735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c63d8-2d5e-406d-8804-7df1b74ef7de None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:22.427254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 582/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:47:22 2022] GET /identity/v3/role_assignments?user.id=9228079f7603433fbfa02953683458cc&scope.project.id=9966aa0955b949f3bffc170f9bc010a8&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:24.891947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d1d89-07d4-4668-9508-628c17b72042 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:24.892531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d1d89-07d4-4668-9508-628c17b72042 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:24.892693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d1d89-07d4-4668-9508-628c17b72042 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:24.894027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 580/1162] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:24.899445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b537-8832-4375-a376-44a0f722f474 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:24.900035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b537-8832-4375-a376-44a0f722f474 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:24.900035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b537-8832-4375-a376-44a0f722f474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:24.906369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:24.919691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9663b537-8832-4375-a376-44a0f722f474 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:24.920946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9663b537-8832-4375-a376-44a0f722f474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:24.987020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 583/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:25.259231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d1d89-07d4-4668-9508-628c17b72042 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:25.359060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d1d89-07d4-4668-9508-628c17b72042 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:25.391479 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:25.392070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:25.392440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:25.392713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:25.441904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:25.457638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:25.459067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:25.554523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 581/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:47:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:47:25.560774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b537-8832-4375-a376-44a0f722f474 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:25.657611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b537-8832-4375-a376-44a0f722f474 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:25.685752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:25.690916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:25.691546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:25.691829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:25.692101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:25.699191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:25.700640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-12ec417d-6fcc-4d6a-b422-6f3d69870f79 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:25.711861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:25.713706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:47:25.714805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 584/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:47:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:25.719781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:25.800744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:25.816980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8f82-5a72-485d-9324-5fb7e833d046 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:25.850171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:25.850766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:25.851048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:25.851321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:25.851952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:25.852253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:25.853640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e380c3f-ddd3-44c7-994e-fe4c70054d27 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:25.860620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 582/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu Sep 15 12:47:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:27.996821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae4af3-22b3-45f0-a7c7-d23fe856e4a7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:27.997374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae4af3-22b3-45f0-a7c7-d23fe856e4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:27.997374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae4af3-22b3-45f0-a7c7-d23fe856e4a7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:27.998695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 585/1167] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:28.003812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d2136-1b3a-45aa-85b9-c1b6e3766bbf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:28.004244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d2136-1b3a-45aa-85b9-c1b6e3766bbf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:28.004404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d2136-1b3a-45aa-85b9-c1b6e3766bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:28.022640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-652d2136-1b3a-45aa-85b9-c1b6e3766bbf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:28.023818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-652d2136-1b3a-45aa-85b9-c1b6e3766bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:28.089346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 583/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:28.370896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae4af3-22b3-45f0-a7c7-d23fe856e4a7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:28.470161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae4af3-22b3-45f0-a7c7-d23fe856e4a7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:28.503112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e95fd9-eb3f-4b9a-a428-d1e638b42c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:28.503702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95fd9-eb3f-4b9a-a428-d1e638b42c7e None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:28.503981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95fd9-eb3f-4b9a-a428-d1e638b42c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:28.504255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e95fd9-eb3f-4b9a-a428-d1e638b42c7e None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:28.504930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e95fd9-eb3f-4b9a-a428-d1e638b42c7e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:28.506279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e95fd9-eb3f-4b9a-a428-d1e638b42c7e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:28.522145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 586/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu Sep 15 12:47:28 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:47:30.680257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7574b-b5a3-457f-bee6-9ff80c83a7cd None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:30.680849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7574b-b5a3-457f-bee6-9ff80c83a7cd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:30.680941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7574b-b5a3-457f-bee6-9ff80c83a7cd None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:30.682754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 584/1170] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:30.687938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:30.688326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:30.688530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:30.706736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:30.707928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:30.754373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:30.774869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 587/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:31.042529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7574b-b5a3-457f-bee6-9ff80c83a7cd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:31.119617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:31.141812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7574b-b5a3-457f-bee6-9ff80c83a7cd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:31.175211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8591120-22fc-4a9a-8cc1-580275d061bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:31.175833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8591120-22fc-4a9a-8cc1-580275d061bd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:31.176113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8591120-22fc-4a9a-8cc1-580275d061bd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:31.176416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8591120-22fc-4a9a-8cc1-580275d061bd None admin] PATH_INFO: `/v3/domains/99cb18c7336e4e5b867b3c3e72797f37` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:31.186444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8591120-22fc-4a9a-8cc1-580275d061bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=99cb18c7336e4e5b867b3c3e72797f37)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:31.187885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8591120-22fc-4a9a-8cc1-580275d061bd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:31.189476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 585/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu Sep 15 12:47:31 2022] GET /identity/v3/domains/99cb18c7336e4e5b867b3c3e72797f37 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:31.195346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:31.292472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5b391-0d58-4512-b54c-1b2d71759c79 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:31.324688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa88fba-4135-44b7-9470-24287b43f98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:31.325222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa88fba-4135-44b7-9470-24287b43f98c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:31.325506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa88fba-4135-44b7-9470-24287b43f98c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:31.325783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa88fba-4135-44b7-9470-24287b43f98c None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:31.326439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa88fba-4135-44b7-9470-24287b43f98c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:31.327853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa88fba-4135-44b7-9470-24287b43f98c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:31.354744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 588/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:47:31 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:47:33.514264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc137f-fad7-472a-8230-32f70b6710ac None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:33.514803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc137f-fad7-472a-8230-32f70b6710ac None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:33.514803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc137f-fad7-472a-8230-32f70b6710ac None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:33.516113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 586/1174] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:33.521274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:33.521716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:33.521856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:33.539769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:33.541131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:33.567276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:33.607773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 589/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:33.876155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc137f-fad7-472a-8230-32f70b6710ac None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:33.974819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc137f-fad7-472a-8230-32f70b6710ac None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:34.007365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839c07c-5bfd-45be-8552-936683d215a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:34.007973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839c07c-5bfd-45be-8552-936683d215a8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:34.008249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839c07c-5bfd-45be-8552-936683d215a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:34.008551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839c07c-5bfd-45be-8552-936683d215a8 None admin] PATH_INFO: `/v3/domains/99cb18c7336e4e5b867b3c3e72797f37` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:34.009995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839c07c-5bfd-45be-8552-936683d215a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=99cb18c7336e4e5b867b3c3e72797f37)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:34.011405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839c07c-5bfd-45be-8552-936683d215a8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:34.013207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 587/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu Sep 15 12:47:33 2022] GET /identity/v3/domains/99cb18c7336e4e5b867b3c3e72797f37 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:34.018843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:34.117197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183fa07-8bd1-4987-a8d6-8ec3bcc5d4ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:34.152011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84334b9f-a4d7-4ed3-b1e8-8d2442f246ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:34.152658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84334b9f-a4d7-4ed3-b1e8-8d2442f246ec None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:34.152944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84334b9f-a4d7-4ed3-b1e8-8d2442f246ec None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:34.153220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84334b9f-a4d7-4ed3-b1e8-8d2442f246ec None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:34.153873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84334b9f-a4d7-4ed3-b1e8-8d2442f246ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:34.155297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84334b9f-a4d7-4ed3-b1e8-8d2442f246ec None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:34.181270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 590/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:47:34 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:47:36.353478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e060-8dcd-41cf-a4c5-b2389443db90 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:36.353989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e060-8dcd-41cf-a4c5-b2389443db90 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:36.354083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593e060-8dcd-41cf-a4c5-b2389443db90 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:36.355333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 588/1178] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:36.360554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:36.360970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:36.361110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:36.378999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:36.380173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:36.394329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:36.447647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 591/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:36.721273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593e060-8dcd-41cf-a4c5-b2389443db90 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:36.730935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:36.820824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593e060-8dcd-41cf-a4c5-b2389443db90 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:36.853920 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:36.854540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:36.854822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:36.855090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:36.865632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:47:36.870744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:36.871862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:47:36.873134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 589/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:47:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:36.877867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:36.974464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987679f3-e888-4ff7-956c-d6ef188ae472 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:37.007405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:37.008005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:37.008299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:37.008579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:37.009910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:37.010213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:37.011571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:37.019644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 592/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:47:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:37.025470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:37.089147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:37.125806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca0165-eae7-4f8a-90d0-a744131a7b1d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:37.158351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:37.158964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:37.159242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:37.159514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] PATH_INFO: `/v3/users/d5f38fc71db74d6786f26bb4613fe26e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:37.172113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] RBAC: Authorizing `identity:get_user(user_id=d5f38fc71db74d6786f26bb4613fe26e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:37.173578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:37.175134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 590/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:37 2022] GET /identity/v3/users/d5f38fc71db74d6786f26bb4613fe26e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:37.181468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:37.197035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:37.280028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23194-db08-4496-9d21-dd6bc665ac78 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:37.313282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:37.313885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:37.314165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:37.314435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:37.378897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:37.380351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:37.395002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:37.480496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 593/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:47:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:47:37.486405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:37.582513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830672-1736-4cb0-9478-28deb82d4b7a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:37.605161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:37.615805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65305e3-c43d-4277-b764-443e564a3f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:37.616528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65305e3-c43d-4277-b764-443e564a3f67 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:37.616807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65305e3-c43d-4277-b764-443e564a3f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:37.617078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65305e3-c43d-4277-b764-443e564a3f67 None admin] PATH_INFO: `/v3/projects/94d0b38f9626416bb81047bfd969071e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:37.618238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65305e3-c43d-4277-b764-443e564a3f67 None admin] RBAC: Authorizing `identity:get_project(project_id=94d0b38f9626416bb81047bfd969071e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:37.619638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65305e3-c43d-4277-b764-443e564a3f67 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:37.621257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 591/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:37 2022] GET /identity/v3/projects/94d0b38f9626416bb81047bfd969071e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:37.627452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:37.726959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00942c-35ca-42e6-8790-eff0e1e519ef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:37.740745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:37.762893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:37.763489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:37.763772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:37.764046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:37.764786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=94d0b38f9626416bb81047bfd969071e, user.id=d5f38fc71db74d6786f26bb4613fe26e) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:37.765093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:37.766504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98db07-d080-4f47-a4b2-c2a0fa9c3a60 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:37.771668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 594/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:47:37 2022] GET /identity/v3/role_assignments?user.id=d5f38fc71db74d6786f26bb4613fe26e&scope.project.id=94d0b38f9626416bb81047bfd969071e&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:39.916586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509fa91-a0b6-4d9f-8fef-2c445f19ca01 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:39.917167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509fa91-a0b6-4d9f-8fef-2c445f19ca01 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:39.917167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509fa91-a0b6-4d9f-8fef-2c445f19ca01 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:39.918452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 592/1186] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:39.923613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:39.924036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:39.924162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:39.942497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:39.943686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:40.009217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 595/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:40.285098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509fa91-a0b6-4d9f-8fef-2c445f19ca01 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:40.384605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509fa91-a0b6-4d9f-8fef-2c445f19ca01 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:40.417958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:40.418606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:40.418881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:40.419148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:40.428956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:47:40.433706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:40.434807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:47:40.435982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 593/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:47:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:40.440857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:40.537873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5761b-7798-44db-8b25-16e812f89622 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:40.544900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:40.570608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:40.571743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:40.572072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:40.572375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:40.573024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:40.573328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:40.574681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:40.582400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 596/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:40.589103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:40.641315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:40.686857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83c126-3503-4bda-a068-0d2f29280c77 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:40.721010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:40.721625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:40.721901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:40.722167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] PATH_INFO: `/v3/users/d5f38fc71db74d6786f26bb4613fe26e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:40.723265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] RBAC: Authorizing `identity:get_user(user_id=d5f38fc71db74d6786f26bb4613fe26e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:40.724719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:40.726261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 594/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:40 2022] GET /identity/v3/users/d5f38fc71db74d6786f26bb4613fe26e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:40.732160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:40.829814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c741-f633-45ea-85b8-8f008d37236f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:40.861501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540baf51-7b87-4859-9127-559b4445d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:40.862117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540baf51-7b87-4859-9127-559b4445d25d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:40.862399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540baf51-7b87-4859-9127-559b4445d25d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:40.862671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540baf51-7b87-4859-9127-559b4445d25d None admin] PATH_INFO: `/v3/projects/94d0b38f9626416bb81047bfd969071e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:40.863767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540baf51-7b87-4859-9127-559b4445d25d None admin] RBAC: Authorizing `identity:get_project(project_id=94d0b38f9626416bb81047bfd969071e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:40.865210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540baf51-7b87-4859-9127-559b4445d25d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:40.866800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 597/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:40 2022] GET /identity/v3/projects/94d0b38f9626416bb81047bfd969071e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:47:40.873113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:40.970312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a699afc-e9c8-4b2a-bfe4-53a84246fc1b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:40.989425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:41.003846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0d4d9-02fb-46f6-baf2-47fbb4698dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:41.004527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0d4d9-02fb-46f6-baf2-47fbb4698dbb None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:41.004811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0d4d9-02fb-46f6-baf2-47fbb4698dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:41.005086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0d4d9-02fb-46f6-baf2-47fbb4698dbb None admin] PATH_INFO: `/v3/projects/94d0b38f9626416bb81047bfd969071e/users/d5f38fc71db74d6786f26bb4613fe26e/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:41.006968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0d4d9-02fb-46f6-baf2-47fbb4698dbb None admin] RBAC: Authorizing `identity:create_grant(project_id=94d0b38f9626416bb81047bfd969071e, user_id=d5f38fc71db74d6786f26bb4613fe26e, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:41.008457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0d4d9-02fb-46f6-baf2-47fbb4698dbb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:41.021354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 595/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:47:40 2022] PUT /identity/v3/projects/94d0b38f9626416bb81047bfd969071e/users/d5f38fc71db74d6786f26bb4613fe26e/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:47:43.128608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417733a-7417-435c-b876-13ef291ee630 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:43.129038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417733a-7417-435c-b876-13ef291ee630 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:43.129147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417733a-7417-435c-b876-13ef291ee630 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:43.130483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 598/1193] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:47:43.135946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:43.136362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:43.136534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:43.155001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:47:43.156176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:43.221126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 596/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:47:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:47:43.488487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417733a-7417-435c-b876-13ef291ee630 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:43.524627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:43.588490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417733a-7417-435c-b876-13ef291ee630 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:43.621537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:43.622158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:43.622440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:43.622713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:43.632993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:47:43.638070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:47:43.639221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:47:43.640205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 599/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:47:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:47:43.645223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:43.740458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561088ee-3573-420e-9660-94982a0d4c1f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:43.750787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:43.773999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:43.775261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:43.775539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:43.775806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:43.776481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:43.776782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:43.778147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:43.785904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 597/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:47:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:43.792584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:43.883870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:43.890376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc077a-36c4-47b5-a8d7-6b6abc46e188 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:43.924025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:43.924672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:43.924956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:43.925232 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] PATH_INFO: `/v3/users/d5f38fc71db74d6786f26bb4613fe26e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:43.926339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] RBAC: Authorizing `identity:get_user(user_id=d5f38fc71db74d6786f26bb4613fe26e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:43.927765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:43.929361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 600/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu Sep 15 12:47:43 2022] GET /identity/v3/users/d5f38fc71db74d6786f26bb4613fe26e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:43.935233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:44.031914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ced76-7ba0-43ef-933f-0ae5c31ec1d3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:44.064445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:44.065035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:44.065314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:44.065584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:44.106380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:44.132484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:44.134541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:44.229538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 598/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:47:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:47:44.235441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:44.332604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5ba0f-d1c9-4957-9c12-15774f83b28b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:44.364544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb8a7-59b2-4161-b25f-83896541ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:44.365157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb8a7-59b2-4161-b25f-83896541ab77 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:44.365434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb8a7-59b2-4161-b25f-83896541ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:44.365703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb8a7-59b2-4161-b25f-83896541ab77 None admin] PATH_INFO: `/v3/projects/94d0b38f9626416bb81047bfd969071e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:44.366795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb8a7-59b2-4161-b25f-83896541ab77 None admin] RBAC: Authorizing `identity:get_project(project_id=94d0b38f9626416bb81047bfd969071e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:44.368436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb8a7-59b2-4161-b25f-83896541ab77 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:44.370037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 601/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu Sep 15 12:47:44 2022] GET /identity/v3/projects/94d0b38f9626416bb81047bfd969071e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:44.376115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:44.445223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:44.474327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea26a7-a0c0-4f18-b46a-626f2a406474 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:44.506991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:44.507587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:44.507864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:44.508134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:44.509611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=94d0b38f9626416bb81047bfd969071e, user.id=d5f38fc71db74d6786f26bb4613fe26e) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:47:44.509918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:44.511342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf9161-dd37-462d-9601-0c8ce91fe897 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:44.517123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 599/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:47:44 2022] GET /identity/v3/role_assignments?user.id=d5f38fc71db74d6786f26bb4613fe26e&scope.project.id=94d0b38f9626416bb81047bfd969071e&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:47:54.360626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:54.514596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=32 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:40.322403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789ca23-900b-4f7f-b346-4b2491864eb9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:40.323044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789ca23-900b-4f7f-b346-4b2491864eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:40.323044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789ca23-900b-4f7f-b346-4b2491864eb9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:40.324498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 602/1201] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:48:40.330220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:40.330879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:40.330879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:40.350230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:48:40.351950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:48:40.440497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 600/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:48:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:48:40.777834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789ca23-900b-4f7f-b346-4b2491864eb9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:40.906387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789ca23-900b-4f7f-b346-4b2491864eb9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:40.947270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:40.947913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:40.948203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:40.948516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:40.961304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:40.964308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:40.986728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:40.987902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:40.988915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:40.988915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:40.988915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:40.993660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 603/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:48:40.994424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:41.117529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb636afd-009c-40f7-900f-985ddab9e4dd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:41.152789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:41.160189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:41.161055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:41.161570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:41.161916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:41.162752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:41.164796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:41.166307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55415b4-63f5-4600-946e-dc1c69201226 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:41.184483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 601/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:48:41.185923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:41.251864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:41.313224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39191fa2-0262-4031-9097-8e626b6abcc4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:41.353950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705147a-a64b-4755-b5b9-c32b31bab0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:41.354780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705147a-a64b-4755-b5b9-c32b31bab0c3 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:41.354986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705147a-a64b-4755-b5b9-c32b31bab0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:41.355259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705147a-a64b-4755-b5b9-c32b31bab0c3 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:41.355905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705147a-a64b-4755-b5b9-c32b31bab0c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:41.357726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705147a-a64b-4755-b5b9-c32b31bab0c3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:41.390498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 604/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu Sep 15 12:48:41 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:48:43.762430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b29e8c-4bfb-4d09-bb22-bc90881a76a0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:43.763265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b29e8c-4bfb-4d09-bb22-bc90881a76a0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:43.763265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b29e8c-4bfb-4d09-bb22-bc90881a76a0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:43.768344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 602/1206] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:48:43.770303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:43.770727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:43.770840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:43.790698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:48:43.791894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:48:43.868659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 605/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:48:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:48:44.146136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b29e8c-4bfb-4d09-bb22-bc90881a76a0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:44.260804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b29e8c-4bfb-4d09-bb22-bc90881a76a0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:44.299581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:44.300214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:44.300537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:44.300811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:44.312224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:48:44.317382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:44.318369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:48:44.319240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:44.324121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 603/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:48:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:44.325141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:44.376196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:44.445768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f73fc-c149-4b04-b2c3-57109462fa40 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:44.482423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:44.483040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:44.483363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:44.483653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:44.484324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:44.484638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:44.488354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:44.494460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 606/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:48:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:44.500480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:44.558652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:44.614525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297cb18-1699-4a99-8e8c-ef568fa19fe3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:44.660455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:44.661018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:44.661298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:44.661567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:44.673584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:44.675077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:44.693835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:44.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:44.695893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:44.695893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:44.695893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:44.700446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 604/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:44.702617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:44.816939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:44.820877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6c817-d83c-44f6-96f9-42ae34b0092d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:44.860660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:44.861432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:44.861714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:44.861989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:44.862648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:44.862998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:44.864480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:44.882762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 607/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:44.883929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:44.998789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1f6d8-5666-447b-8391-6c5a659ed299 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:45.000751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:45.043761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:45.044474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:45.044792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:45.045298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:45.053822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:45.055476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:48:45.068222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:45.070048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:45.071448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:45.076355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:45.076552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 605/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:45.196700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097157c-85bd-4912-8bd3-adac215b2705 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:45.238631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:45.239272 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:45.239558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:45.239836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:45.240574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:45.240881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:45.243285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:45.267366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 608/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Sep 15 12:48:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:45.268478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:45.381153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165113-43ae-4d02-a842-1c9388a74e66 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:45.427596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:45.428573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:45.429759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:45.430207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:45.474297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:45.521018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:45.522441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:45.640092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 606/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:48:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:48:45.645774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:45.666326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:45.759383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803845a-bf9f-4fc9-ad65-f3b686f62d3b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:45.796833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:45.797382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:45.797668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:45.797928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:45.809765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:45.811485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:45.827395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:45.829059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:45.829939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:45.829939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:45.829939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:45.835034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:45.836372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 609/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:45.903316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:45.954174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e54cc-e746-41c6-9372-3e8f58112488 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:45.993946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:45.994851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:45.995186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:45.995634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:45.996300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:45.996648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:45.998409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:46.016853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 607/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:46.018126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:46.107694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:46.150651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ad3f5-4d2e-4dfd-b7bf-161e01d6e468 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:46.200125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:46.200799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:46.201086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:46.201363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:46.208454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:46.210181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:48:46.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:46.223469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:48:46.224510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:46.229970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 610/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:48:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:46.230126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:46.354583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafcb4e-d89f-423d-b40a-7fbed158bed1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:46.381783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:46.394907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:46.395600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:46.395957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:46.396321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:46.397088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:46.397606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:46.399017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85663b-05a8-49c5-8ddc-4a104c40ba9c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:46.412182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 608/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:46.419207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:46.542340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e825fb-282e-44da-a87a-b8f6b60b9d24 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:46.582176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:46.587284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:46.587874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:46.588153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:46.588482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:46.590206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=25816463963b44a8865c0d3693cfef88) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:46.590565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:46.594596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e285dd5-eb5f-42d3-99d6-04311e110b69 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:46.598185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 611/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:48:46 2022] GET /identity/v3/role_assignments?user.id=25816463963b44a8865c0d3693cfef88&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:48.769519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43ac1-68de-4e71-b756-3be46d11efc0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:48.770053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43ac1-68de-4e71-b756-3be46d11efc0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:48.770053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e43ac1-68de-4e71-b756-3be46d11efc0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:48.771392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 609/1220] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:48:48.777284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:48.777706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:48.777842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:48.796325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:48:48.799212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:48:48.873097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 612/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:48:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:48:49.160443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43ac1-68de-4e71-b756-3be46d11efc0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:49.266261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e43ac1-68de-4e71-b756-3be46d11efc0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:49.302257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:49.302906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:49.303170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:49.303445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:49.315983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:48:49.321539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:49.322665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:49.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 610/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:48:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:49.329072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:49.436648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb7713-62c2-4331-90b0-0de450d52153 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:49.462442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:49.477212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:49.477899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:49.478358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:49.478691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:49.479521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:49.479832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:49.481342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:49.490513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 613/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:48:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:48:49.496465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:49.553480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:49.612669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0841f3-3eed-4fa7-9c5b-7152c3afbbbd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:49.648671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:49.649308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:49.649593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:49.649866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:49.660334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:49.661742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:49.677039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:49.678214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:49.679454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:49.679454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:49.679454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:49.679454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 611/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:49.684515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:49.802440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e21596-fffe-4ca6-b5a0-66e75c2d4aab None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:49.841112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:49.841727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:49.842010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:49.842285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:49.842938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:49.843242 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:49.844849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:49.857853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 614/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:49.864654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:49.971879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:49.979296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de41db8-db13-4809-af87-ad44d3b95b44 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:50.014951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:50.015572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:50.015846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:50.016133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:50.024000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:50.025471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:48:50.036463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:50.037652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:50.038785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 612/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:48:50.045634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:50.099373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:50.158452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4441db-4c1e-44ca-bdb9-5c2e1dab650c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:50.197448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:50.198059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:50.198460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:50.198759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:50.199508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:50.199837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:50.201474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:50.227031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 615/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Sep 15 12:48:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:50.228245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:50.336591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7c5e6-52b7-4350-8cc6-605a91c02006 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:50.372680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:50.373245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:50.373597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:50.373981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:50.386575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:50.388041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:50.404839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:50.405869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:50.406751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:50.406751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:50.406751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:50.406751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 613/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:50.413033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:50.532071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c12822-486c-450e-880d-6af8f9cfc9c2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:50.534486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:50.574166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:50.574776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:50.575057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:50.575328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:50.575998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:50.581699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:50.581699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:50.590676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 616/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:50.596594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:50.606150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:50.702344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a092c-fc91-4606-a4a6-09eae08a860c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:50.747606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:50.748506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:50.748800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:50.749163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:50.758359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:50.759952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:48:50.773217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:50.774344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:50.775502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 614/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:50.780443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:50.887422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a6f6c-01a5-44a3-b89e-ffceca06b5b0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:50.923526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:50.924485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:50.924772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:50.925044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:50.925823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:50.926136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:50.927747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b2703-9533-4483-97a2-168df6a7e90d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:50.941410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 617/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:48:50.947721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:51.022275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:51.065882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670df0-70b1-4734-a458-184fe2f082ad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:51.103414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede229e1-f5a0-4ce1-8175-e3f015e28503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:51.104075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede229e1-f5a0-4ce1-8175-e3f015e28503 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:51.104385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede229e1-f5a0-4ce1-8175-e3f015e28503 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:51.104655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede229e1-f5a0-4ce1-8175-e3f015e28503 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/25816463963b44a8865c0d3693cfef88/roles/2985674d70594ed99a4dfb02df10718c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:51.119742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede229e1-f5a0-4ce1-8175-e3f015e28503 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=25816463963b44a8865c0d3693cfef88, role_id=2985674d70594ed99a4dfb02df10718c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:51.121227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede229e1-f5a0-4ce1-8175-e3f015e28503 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:51.136815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 615/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:48:50 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/25816463963b44a8865c0d3693cfef88/roles/2985674d70594ed99a4dfb02df10718c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:48:53.316884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c54842-8122-40b9-9d74-d82da6f5f24d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:53.317541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c54842-8122-40b9-9d74-d82da6f5f24d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:53.317541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c54842-8122-40b9-9d74-d82da6f5f24d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:53.318962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 618/1233] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:48:53.325036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:53.325595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:53.325595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:53.345023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:48:53.346511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:48:53.416617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 616/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:48:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:48:53.708236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c54842-8122-40b9-9d74-d82da6f5f24d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:53.742160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:53.831747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c54842-8122-40b9-9d74-d82da6f5f24d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:53.874482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:53.875132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:53.875477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:53.875779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:53.889286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:48:53.894633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:53.896697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:53.899286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 619/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu Sep 15 12:48:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:53.905280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:53.957050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:54.017094 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11871-41a7-466c-a295-86b4b91af7ab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:54.058847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:54.059561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:54.059908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:54.060201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:54.060902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:54.061207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:54.062577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:54.071516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 617/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu Sep 15 12:48:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:48:54.077713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:54.173970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:54.192231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9593dd-9bd0-415f-b4b6-bab2f2e2eec5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:54.231348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:54.232060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:54.232375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:54.232675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:54.244571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:54.246028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:54.265843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:54.267235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:54.268464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:54.268464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:54.268464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:54.268464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 620/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:48:54.275039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:54.391045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379bda0-24e8-4026-8f4c-d1e3eb863173 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:54.402849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:54.432489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:54.433101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:54.433392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:54.433694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:54.434514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:54.434831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:54.436498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:54.449822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 618/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:54.457226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:54.565224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c0cfb-4ddd-4502-b078-53e249f775fe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:54.600074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:54.601592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:54.601885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:54.602150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:54.604943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:54.611122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:54.612725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:48:54.624304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:54.625460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:54.626512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 621/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:54.632590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:54.741624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb11e2f-5a68-429a-a9c9-6b136563510d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:54.781914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:54.782516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:54.782792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:54.783060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:54.783778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:54.784079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:54.785539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:54.804662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 619/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Sep 15 12:48:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:54.811339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:54.920498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769de11d-b9b5-40ab-87d6-9e4c0ad743e4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:54.957123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:54.957987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:54.958460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:54.958731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:55.028121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:55.035206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:55.036637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:55.146813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 622/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:48:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:48:55.152795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:55.182158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:55.264770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750e9e2-53f1-4d01-b29c-3b67ae7a0b7b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:55.301447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:55.302101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:55.302382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:55.302661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:55.313799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:55.315383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:55.332509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:55.333820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:55.335162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:55.335162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:55.335162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:55.335162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 620/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:55.340000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:55.442631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:55.450258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27061432-2742-4f43-9b06-9e0a790728fd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:55.486912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:55.487662 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:55.487807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:55.488086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:55.488815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:55.489192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:55.490709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:55.503713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 623/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:55.510403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:55.593529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:55.617017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c068f9-cbb8-41cf-ab23-382ccc1b5c90 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:55.654207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:55.654851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:55.655144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:55.655403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:55.663864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:55.665775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:48:55.679417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:55.680627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:48:55.681701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:48:55.682229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 621/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:55.687175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:55.806320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a48bb-6b46-4376-8291-da0972586130 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:55.842294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:55.842912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:55.843192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:55.843464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:55.844148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:55.844486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:55.845920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790d05-71c9-4566-b67a-54fe00fb88a6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:55.859097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 624/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:48:55.865098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:55.981447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed751566-4183-40d0-94b8-e0f5eacf9677 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:56.019545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:56.020152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:56.020462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:56.020759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:56.021449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] RBAC: Adding query filter params (role.id=2985674d70594ed99a4dfb02df10718c, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=25816463963b44a8865c0d3693cfef88) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:56.021760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:56.023425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fd61b-0143-4ffd-8053-f0d328fd2270 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:56.034031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 622/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:48:55 2022] GET /identity/v3/role_assignments?user.id=25816463963b44a8865c0d3693cfef88&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=2985674d70594ed99a4dfb02df10718c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:58.396086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403c53-786c-4aa0-9f14-70132b09a36d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:58.396812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403c53-786c-4aa0-9f14-70132b09a36d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:58.396812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403c53-786c-4aa0-9f14-70132b09a36d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:58.398093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 625/1247] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:48:58.403311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:58.403740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:58.403864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:58.410283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:58.423763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:48:58.425002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:48:58.493655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 623/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:48:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:48:58.781703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1403c53-786c-4aa0-9f14-70132b09a36d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:58.802100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:58.916677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1403c53-786c-4aa0-9f14-70132b09a36d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:58.962139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:58.962770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:58.963073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:58.963355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:58.973549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:48:58.979340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:58.980911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:58.982360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 626/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:58.986903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:59.103949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954957f6-9c40-4fde-8230-6be2984b703f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:59.139234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:59.145299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:59.146201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:59.146674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:59.147079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:59.148487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:59.148853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:59.150432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:59.159403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 624/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:48:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:59.165908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:59.244993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:59.270834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66ec43-6063-438d-a347-dfa40a021787 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:59.307629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:59.308258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:59.308569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:59.308840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:59.320096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:59.321567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:48:59.338483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:59.339552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:48:59.340920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:48:59.340920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:48:59.340920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:59.340920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 627/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:59.345427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:59.452406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7c0a-0842-4c5f-88eb-8b00f6ea5504 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:59.488920 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:59.489722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:59.490039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:59.490355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:59.491157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:59.491497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:59.493084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:59.506969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 625/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:59.512835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:59.625058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f80f37-7cf3-4daf-bf00-bf07f61a1e81 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:59.654572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:59.664531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:59.665164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:59.666865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:59.666865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:59.673378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:59.674898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:48:59.686639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:48:59.687841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:48:59.689007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:48:59.692491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 628/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:48:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:48:59.693660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:48:59.766550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:59.823324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094a619-dfa5-4e15-be5b-e9e2a8d7cdc5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:48:59.862149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:48:59.862760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:48:59.863053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:48:59.863313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:48:59.864058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:48:59.864380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:48:59.865886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:48:59.884157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 626/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Sep 15 12:48:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:48:59.890275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:00.001050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66e0e-34d4-4fbc-8880-267e016873ae None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:00.039754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:00.040460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:00.040749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:00.041023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:00.104619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:00.118310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:00.119797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:00.225374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 629/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:48:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:49:00.231298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:00.335833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02aed87-27aa-460f-b1db-31ac618649be None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:00.350012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:00.373018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:00.373670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:00.373951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:00.374224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:00.385723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:00.387192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:49:00.402416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:00.403428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:49:00.404754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:49:00.404754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:00.404754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:00.404754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 627/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:00.409313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:00.500665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:00.514608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426048a1-4b14-4449-9812-445e2bcb5ae0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:00.549883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:00.550499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:00.550781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:00.551051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:00.551684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:00.551990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:00.554058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:00.565750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 630/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:00.572138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:00.676845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d86ea-1ba5-43f7-aece-af6f1a63a486 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:00.710316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:00.711298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:00.711604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:00.712065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:00.720742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:00.722740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:49:00.740580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:00.742027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:00.742927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 628/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:00.750026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:00.849295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcccbd-0961-494e-86a0-2c82a9240eee None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:00.883018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:00.883615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:00.883894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:00.884164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:00.884872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:00.885177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:00.886587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786aaf5-b632-4a74-a3c7-23a25c160b02 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:00.898541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 631/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:49:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:00.904632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:00.930483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:01.002435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079e799-99f1-420f-a6b0-32536653d47f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:01.035057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:01.035652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:01.035937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:01.036208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:01.036951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=25816463963b44a8865c0d3693cfef88) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:01.037261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:01.039592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694666bb-5fbc-4b0a-b442-a18edbd7decf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:01.044784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 629/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:49:00 2022] GET /identity/v3/role_assignments?user.id=25816463963b44a8865c0d3693cfef88&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=13d4313b2fe74b3688999f3077afce5d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:03.192731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb704b6b-13de-4eec-9ecb-b6c78bd34ea6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:03.193241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb704b6b-13de-4eec-9ecb-b6c78bd34ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:03.193367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb704b6b-13de-4eec-9ecb-b6c78bd34ea6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:03.194613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 632/1261] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:49:03.200225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:03.200654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:03.200807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:03.219078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:49:03.220261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:49:03.285208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 630/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:49:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:49:03.556794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb704b6b-13de-4eec-9ecb-b6c78bd34ea6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:03.567578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:03.657754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb704b6b-13de-4eec-9ecb-b6c78bd34ea6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:03.689965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:03.690582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:03.690865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:03.691135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:03.701564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:49:03.706430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:03.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:03.708693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 633/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:03.715845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:03.735772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:03.819688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a196-e028-4fea-ac4f-4b39838e5d20 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:03.852793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:03.853391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:03.853667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:03.853934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:03.854581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:03.854876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:03.856236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:03.864008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 631/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:03.869905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:03.936449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:03.968993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9d20e-1af5-4322-b645-08865c0e1321 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.001813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.002420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.002698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.002967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.013713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.015131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:49:04.029327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:04.030424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:49:04.031306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:49:04.031306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:04.031306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:04.031306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 634/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:04.036117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:04.118721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:04.137824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3c59-b404-4635-90b6-c312a3a049bd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.171023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.171621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.171899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.172172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.172833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:04.173137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.174501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.185768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 632/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:04.191429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:04.291246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433d56e-e569-4e94-a7e9-886cc141c579 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.310059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:04.324621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.325244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.325528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.325802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.333012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.334443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:49:04.345971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:04.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:04.348088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 635/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:04.353051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:04.449964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13682881-7c54-4822-b9aa-d6122c3cb950 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.482667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.483265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.483577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.483850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.484605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:04.484910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.486327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.494662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:04.503732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 633/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:04.509997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:04.607236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400c8e1-cbc1-40fa-9650-ebc414931365 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.645999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.646618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.646897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.647168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.658312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.659743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:49:04.673850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:04.674973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:49:04.676209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:49:04.676209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:04.676209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:04.676209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 636/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:04.681300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:04.784767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44717ef5-43bb-4f52-98fd-89b2c07fa0fc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.817283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.817979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.818256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.818519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.819139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:04.819435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.821853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.832533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 634/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:49:04.838601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:04.869999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:04.936729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eee5e2-6a3c-422e-8270-04372e5a5386 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:04.970013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:04.970639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:04.970920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:04.971192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:04.977827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:04.979262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:49:04.990229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:04.991248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:04.992877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 637/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:04.997800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:05.054875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:05.096663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ca70b-3d49-4c5d-9fbf-b08b4116788f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:05.129807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:05.130346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:05.130627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:05.130902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:05.131586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:05.131892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:05.133389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4631a-360d-47f9-8308-2ece6f18cdc3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:05.145229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 635/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:49:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:05.151254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:05.239397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:05.249104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9e909-7ccd-4c9c-b5ef-597ec2ccab54 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:05.281561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf099c7-7b9f-41df-a920-be5b3555d4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:05.282204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf099c7-7b9f-41df-a920-be5b3555d4a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:05.282484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf099c7-7b9f-41df-a920-be5b3555d4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:05.282755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf099c7-7b9f-41df-a920-be5b3555d4a4 None admin] PATH_INFO: `/v3/projects/99e6a332ee23460796823d362d81e7bd/users/25816463963b44a8865c0d3693cfef88/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:05.284648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf099c7-7b9f-41df-a920-be5b3555d4a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=99e6a332ee23460796823d362d81e7bd, user_id=25816463963b44a8865c0d3693cfef88, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:05.286064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf099c7-7b9f-41df-a920-be5b3555d4a4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:05.298728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 638/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu Sep 15 12:49:05 2022] PUT /identity/v3/projects/99e6a332ee23460796823d362d81e7bd/users/25816463963b44a8865c0d3693cfef88/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:49:07.423174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4c394-bb6e-4dce-9470-744943dbae9d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:07.423694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4c394-bb6e-4dce-9470-744943dbae9d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:07.423694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4c394-bb6e-4dce-9470-744943dbae9d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:07.425002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 636/1274] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:49:07.430527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:07.430935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:07.431079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:07.449548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:49:07.450727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:49:07.515683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 639/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:49:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:49:07.796426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4c394-bb6e-4dce-9470-744943dbae9d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:07.896572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4c394-bb6e-4dce-9470-744943dbae9d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:07.914363 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:07.931108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:07.931733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:07.932014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:07.932310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:07.942106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:49:07.946883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:07.947990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:07.949073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 637/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu Sep 15 12:49:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:07.954104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:08.034558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:08.053184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1d927-2d6b-4850-af5a-dfa1236423bf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:08.089221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:08.089838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:08.090118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:08.090389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:08.091045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:08.091347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:08.092775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:08.100980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 640/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu Sep 15 12:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:08.106750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:08.203570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf408281-1ca6-468d-ba60-c88b80d8aac7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:08.236506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:08.237124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:08.237408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:08.237684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:08.247782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:08.249297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:49:08.264388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:08.265584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:49:08.266508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:49:08.266508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:08.266508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:08.266508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 638/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:08.271177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:08.369394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf20708-bf3d-45fb-8075-b1215dd991d2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:08.403001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:08.403599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:08.403882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:08.404154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:08.404816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:08.405121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:08.406474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:08.417873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 641/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:08.423465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:08.455023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:08.522936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012adf34-d310-480d-a906-a80d2144d813 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:08.555432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:08.556057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:08.556364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:08.556641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:08.563763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:08.565223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:49:08.576663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:08.577739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:08.579022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 639/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:08.583448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:08.587249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:08.687015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cec00-2749-4ee6-8256-672bfd987a85 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:08.720850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:08.721450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:08.721734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:08.722007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:08.722713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:08.723052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:08.724467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:08.741324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 642/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu Sep 15 12:49:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:08.747757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:08.837210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:08.846998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b162149-57e1-4088-b262-cc4e7a9e5a5e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:08.880025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:08.880642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:08.880917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:08.881184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:08.945915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:08.947272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:09.032467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:09.043460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 640/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu Sep 15 12:49:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:49:09.049405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:09.129094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:09.173037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7060f19-ecdf-4091-8424-9baed7c27ad9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:09.211172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:09.211813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:09.212095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:09.212397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:09.222676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:09.224174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:49:09.240732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:09.241726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:49:09.242847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:49:09.242847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:49:09.242847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:09.242847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 643/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:09.247966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:09.347613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9ff48-ac92-4509-b1ea-8a8db2dd5e92 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:09.384202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:09.384824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:09.385101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:09.385371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:09.386008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:09.386310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:09.387671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:09.400623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 641/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu Sep 15 12:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:49:09.405799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:09.507319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ad2c9-cd83-4803-a7cc-2fab42933244 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:09.520972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:09.542385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:09.543026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:09.543317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:09.543603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:09.550182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:09.551611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:49:09.562724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:09.563796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 15 12:49:09.564923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 644/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu Sep 15 12:49:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:09.570230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:09.632840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:09.693559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7f76e-de6f-43ff-a876-8bc583399a22 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:09.728255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:09.728879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:09.729153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:09.729420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:09.730092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:09.730390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:09.731816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d7343-0ff2-4784-9207-0f6d8a1062ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:09.744184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 642/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu Sep 15 12:49:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:09.750345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:09.857980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd99682-15cc-499f-82b8-eef67bbdf004 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:09.891502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:09.892103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:09.892416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:09.892690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:09.893401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] RBAC: Adding query filter params (role.id=13d4313b2fe74b3688999f3077afce5d, scope.project.id=99e6a332ee23460796823d362d81e7bd, user.id=25816463963b44a8865c0d3693cfef88) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:09.893708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:09.895129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508e854-5314-43ea-b046-ee2bfdf6b7f0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:09.901701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 645/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu Sep 15 12:49:09 2022] GET /identity/v3/role_assignments?user.id=25816463963b44a8865c0d3693cfef88&scope.project.id=99e6a332ee23460796823d362d81e7bd&role.id=13d4313b2fe74b3688999f3077afce5d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:12.097162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374119-95a5-4269-89e9-a6c15760b2bc None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:12.097587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374119-95a5-4269-89e9-a6c15760b2bc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:12.097724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b374119-95a5-4269-89e9-a6c15760b2bc None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:12.099058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 643/1288] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:49:12.104798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:12.105196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:12.105349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:12.124179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:49:12.125389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:49:12.155325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:12.192250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 646/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:49:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:49:12.468844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374119-95a5-4269-89e9-a6c15760b2bc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:12.567003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b374119-95a5-4269-89e9-a6c15760b2bc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:12.578080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:12.601867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:12.602474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:12.602751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:12.603021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:12.603586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:12.604968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:12.613355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:12.614552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:12.615597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 644/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:49:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:12.620379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:12.719649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c6c2a-e472-4406-a1cd-e264c9bfc9ab None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:12.759892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:12.760533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:12.760813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:12.761197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:12.761953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:12.762290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:12.764204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9463f2-34fd-467d-8f5c-4bdcde818179 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:12.769438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 647/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Sep 15 12:49:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:12.776420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:12.875503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0642fbe-f218-4b07-9b27-83c9a7e071a1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:12.909088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:12.909687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:12.909963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:12.910229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:12.910846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:12.911143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:12.912525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7df9c-6ed6-45ee-95a4-93c0258bd8a0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:12.917097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 645/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Sep 15 12:49:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:15.048112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d2c3-57a7-44e4-b85d-b14bf18e062c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:15.048683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d2c3-57a7-44e4-b85d-b14bf18e062c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:15.048683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d2c3-57a7-44e4-b85d-b14bf18e062c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:15.050012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 648/1293] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:49:15.055473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f302-bbd0-4e02-b84d-017ee2ae5efb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:15.055870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f302-bbd0-4e02-b84d-017ee2ae5efb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:15.056027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f302-bbd0-4e02-b84d-017ee2ae5efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:15.074359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d521f302-bbd0-4e02-b84d-017ee2ae5efb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:49:15.075547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d521f302-bbd0-4e02-b84d-017ee2ae5efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:49:15.104941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:15.142148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 646/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:49:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:49:15.411547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d2c3-57a7-44e4-b85d-b14bf18e062c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:15.442327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:15.511123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d2c3-57a7-44e4-b85d-b14bf18e062c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:15.544044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169e480-2728-4736-9b5e-bdefbac44abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:15.544667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e480-2728-4736-9b5e-bdefbac44abf None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:15.544950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e480-2728-4736-9b5e-bdefbac44abf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:15.545224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169e480-2728-4736-9b5e-bdefbac44abf None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:15.545869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169e480-2728-4736-9b5e-bdefbac44abf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:15.547220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169e480-2728-4736-9b5e-bdefbac44abf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:15.561626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 649/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu Sep 15 12:49:15 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:49:17.699988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d911b3b-a6d3-479b-871d-153e6f577836 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:17.700580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d911b3b-a6d3-479b-871d-153e6f577836 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:17.700580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d911b3b-a6d3-479b-871d-153e6f577836 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:17.701901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 647/1296] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:49:17.707167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:17.707541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:17.707734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:17.729960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:49:17.731281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:49:17.815404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 650/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:49:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:49:18.084007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d911b3b-a6d3-479b-871d-153e6f577836 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:18.182449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d911b3b-a6d3-479b-871d-153e6f577836 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:18.216027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:18.216690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:18.216975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:18.217249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:18.217830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:18.219205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:18.228207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:18.232446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:18.233534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 648/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:49:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:18.235262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:18.253553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:18.335303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1951c-d4d8-4cf0-9396-e22f766b569a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:18.368481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:18.369076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:18.369352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:18.369620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:18.370230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] RBAC: Adding query filter params (name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:18.370527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:18.371873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:18.376805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 651/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Sep 15 12:49:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:18.383373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:18.414047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:18.484813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b18aa-4833-4f3b-b3fb-c0c32962bab7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:18.517950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:18.518544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:18.518822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:18.519091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:18.519782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] RBAC: Adding query filter params (interface=public, service_id=0b1681cf450a4fc7aa7eeafffc62c1c9, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:18.520088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:18.521480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b653b625-66a8-4638-81cb-136eddbcf319 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:18.526321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 649/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu Sep 15 12:49:18 2022] GET /identity/v3/endpoints?service_id=0b1681cf450a4fc7aa7eeafffc62c1c9&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:18.531908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:18.610583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:18.628803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98b5f-3342-4dd8-9e64-0436491ceede None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:18.663680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163399-f243-4901-87bd-0af133aee601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:18.664310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163399-f243-4901-87bd-0af133aee601 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:18.664584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163399-f243-4901-87bd-0af133aee601 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:18.664858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163399-f243-4901-87bd-0af133aee601 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:18.665454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35163399-f243-4901-87bd-0af133aee601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:18.666803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35163399-f243-4901-87bd-0af133aee601 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:18.673506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 652/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu Sep 15 12:49:18 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:49:20.801335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb949-8515-41cf-ba71-45b7b46a757b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:20.801916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb949-8515-41cf-ba71-45b7b46a757b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:20.801916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bb949-8515-41cf-ba71-45b7b46a757b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:20.803219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 650/1302] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:49:20.808844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:20.809236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:20.809402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:20.827818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:49:20.829035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:49:20.894416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 653/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:49:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:49:21.164405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bb949-8515-41cf-ba71-45b7b46a757b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:21.257692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:21.264760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bb949-8515-41cf-ba71-45b7b46a757b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:21.297371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:21.297991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:21.298274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:21.298547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:21.299117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:21.300507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:49:21.309720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:49:21.312452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 15 12:49:21.313531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 651/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu Sep 15 12:49:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:49:21.316744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:21.391025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:21.414781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f020ab1-1272-49fb-8bc8-755df4e01d40 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:21.447312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:21.447919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:21.448202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:21.448509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:21.449127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:49:21.449429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:21.450785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640ee7d-0200-43d9-9261-232e5d69bd89 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:21.455721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 654/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu Sep 15 12:49:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:49:21.462116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:49:21.559284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db30ea0-40aa-42cd-9f3b-f111ff97833a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:49:21.592979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5f66a-5151-441e-a83f-0d3316ab5e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:49:21.593571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f66a-5151-441e-a83f-0d3316ab5e8b None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:49:21.593851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f66a-5151-441e-a83f-0d3316ab5e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:49:21.594119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5f66a-5151-441e-a83f-0d3316ab5e8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:49:21.594779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5f66a-5151-441e-a83f-0d3316ab5e8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:49:21.596162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5f66a-5151-441e-a83f-0d3316ab5e8b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:49:21.614777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 652/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:49:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:49:31.452878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=30 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:49:31.610970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:25.756877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f5e29-98ad-4506-b6ba-2101407c460e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:25.757345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f5e29-98ad-4506-b6ba-2101407c460e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:25.757433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f5e29-98ad-4506-b6ba-2101407c460e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:25.759594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 655/1307] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:25.764865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b34d0-f45f-4c46-b2c4-65e81889f45b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:25.765289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b34d0-f45f-4c46-b2c4-65e81889f45b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:25.765551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b34d0-f45f-4c46-b2c4-65e81889f45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:25.917496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e42b34d0-f45f-4c46-b2c4-65e81889f45b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:25.918713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e42b34d0-f45f-4c46-b2c4-65e81889f45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:26.149201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 653/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:26.447757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c4819-5ae7-4fef-b59c-9622da9703f3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:26.448379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c4819-5ae7-4fef-b59c-9622da9703f3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:26.448815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c4819-5ae7-4fef-b59c-9622da9703f3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:26.450574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 656/1309] 10.222.0.57 () {54 vars in 965 bytes} [Thu Sep 15 12:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:26.456793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456720be-d9f2-497f-8da6-ade126647803 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:26.457241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456720be-d9f2-497f-8da6-ade126647803 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:26.457608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456720be-d9f2-497f-8da6-ade126647803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:26.544084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-456720be-d9f2-497f-8da6-ade126647803 None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:26.545507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-456720be-d9f2-497f-8da6-ade126647803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:26.571685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:26.651823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 654/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Thu Sep 15 12:50:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:26.660992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c4819-5ae7-4fef-b59c-9622da9703f3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:26.664146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-356c4819-5ae7-4fef-b59c-9622da9703f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:26.803205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c4819-5ae7-4fef-b59c-9622da9703f3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:26.892429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:26.893052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:26.893396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:26.893677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:26.895439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:26.962953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:27.026536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:27.027878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7df506-fc4a-463e-b55c-f362ae0c5d83 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:27.048994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 657/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:29.602859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcf9c0-05b0-4c62-914e-111018e0c24b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:29.603304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcf9c0-05b0-4c62-914e-111018e0c24b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:29.603503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcf9c0-05b0-4c62-914e-111018e0c24b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:29.604782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 655/1312] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:29.610101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5454f7-d613-4d53-802e-ef4a8ff688d8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:29.610481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5454f7-d613-4d53-802e-ef4a8ff688d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:29.610690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5454f7-d613-4d53-802e-ef4a8ff688d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:29.692130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bd5454f7-d613-4d53-802e-ef4a8ff688d8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:29.693432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd5454f7-d613-4d53-802e-ef4a8ff688d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:29.789030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 658/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:30.074396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcf9c0-05b0-4c62-914e-111018e0c24b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:30.075958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d3bcf9c0-05b0-4c62-914e-111018e0c24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:30.191871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcf9c0-05b0-4c62-914e-111018e0c24b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:30.268886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:30.271210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:30.272539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:30.272828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:30.273288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:30.275124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:30.383449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:30.384874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63dfcc-ebda-4cdb-9c77-114fe5b94fcc service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:30.405669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 656/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:32.913984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757b0a-73e3-422c-9d66-bec56a952be6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:32.914551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757b0a-73e3-422c-9d66-bec56a952be6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:32.914551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2757b0a-73e3-422c-9d66-bec56a952be6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:32.915875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 659/1315] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:32.921050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2504b0b-de0e-4d3c-8d80-752213255352 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:32.921457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2504b0b-de0e-4d3c-8d80-752213255352 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:32.921598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2504b0b-de0e-4d3c-8d80-752213255352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:32.986936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e2504b0b-de0e-4d3c-8d80-752213255352 None None] MFA Rules not processed for user `2921a61a62e3462084377c936b42cb02`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:32.988128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2504b0b-de0e-4d3c-8d80-752213255352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:33.040470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:33.085577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 657/1316] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:33.370341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2757b0a-73e3-422c-9d66-bec56a952be6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:33.371870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f2757b0a-73e3-422c-9d66-bec56a952be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:33.381780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:33.473914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2757b0a-73e3-422c-9d66-bec56a952be6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:33.547808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:33.548440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:33.548719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:33.548986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:33.550550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:33.612004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:33.658765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:33.660102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b50322-a0b2-4581-b6f8-185d1f4c6299 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:33.678681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 660/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:33 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:36.369337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf8660-ee27-4ca5-a209-d9f9b3553018 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:36.369853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf8660-ee27-4ca5-a209-d9f9b3553018 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:36.369853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf8660-ee27-4ca5-a209-d9f9b3553018 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:36.371117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 658/1318] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:50:36.376292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bc025-2de0-4738-84b7-b8d9a21a1175 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:36.376735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bc025-2de0-4738-84b7-b8d9a21a1175 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:36.376864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57bc025-2de0-4738-84b7-b8d9a21a1175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:36.442067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a57bc025-2de0-4738-84b7-b8d9a21a1175 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:36.443896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a57bc025-2de0-4738-84b7-b8d9a21a1175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:36.539298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 661/1319] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:36.841811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be4e6f-95a9-41e7-8774-697517c79b3c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:36.842097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be4e6f-95a9-41e7-8774-697517c79b3c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:36.842374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be4e6f-95a9-41e7-8774-697517c79b3c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:36.843707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 659/1320] 10.222.0.57 () {54 vars in 965 bytes} [Thu Sep 15 12:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:36.849604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0309f-235d-4b9a-af27-b04c546207cd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:36.849971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0309f-235d-4b9a-af27-b04c546207cd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:36.850249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0309f-235d-4b9a-af27-b04c546207cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:36.872524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:36.926351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-94b0309f-235d-4b9a-af27-b04c546207cd None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:36.928076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94b0309f-235d-4b9a-af27-b04c546207cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:37.004026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 662/1321] 10.222.0.57 () {58 vars in 1062 bytes} [Thu Sep 15 12:50:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:37.013813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be4e6f-95a9-41e7-8774-697517c79b3c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:37.015622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-84be4e6f-95a9-41e7-8774-697517c79b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:37.117720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be4e6f-95a9-41e7-8774-697517c79b3c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:37.191097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:37.191701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:37.191979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:37.192250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:37.194089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:37.200926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:37.303300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:37.304776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6f4f1-69a2-498b-baf4-9af632b75904 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:37.322734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 660/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:40.594084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ba319-b500-4969-a553-d8ac4dabfc16 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:40.594611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ba319-b500-4969-a553-d8ac4dabfc16 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:40.594611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ba319-b500-4969-a553-d8ac4dabfc16 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:40.602371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a363-0a4f-4f59-9fe9-a432e557736b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:40.602931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a363-0a4f-4f59-9fe9-a432e557736b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:40.602931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2a363-0a4f-4f59-9fe9-a432e557736b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:40.607857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 663/1323] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:40.679007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-83d2a363-0a4f-4f59-9fe9-a432e557736b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:40.680252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-83d2a363-0a4f-4f59-9fe9-a432e557736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:40.746030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:40.789057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 661/1324] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:41.075559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ba319-b500-4969-a553-d8ac4dabfc16 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:41.077150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-845ba319-b500-4969-a553-d8ac4dabfc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:41.187457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ba319-b500-4969-a553-d8ac4dabfc16 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:41.200417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:41.262700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:41.263319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:41.263603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:41.263878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:41.265775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:41.374548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:41.375990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc500f-3736-4f93-bf23-a2dd27d4e6d1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:41.394714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 664/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:43.616326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d834a6-7a9a-435d-9e61-d5722626e3c6 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:43.616880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d834a6-7a9a-435d-9e61-d5722626e3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:43.616880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d834a6-7a9a-435d-9e61-d5722626e3c6 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:43.618359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 662/1326] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:43.623452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6786e-5864-4792-8b72-9a71fd65d3f8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:43.623850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6786e-5864-4792-8b72-9a71fd65d3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:43.624015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6786e-5864-4792-8b72-9a71fd65d3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:43.674241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:43.693751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-aac6786e-5864-4792-8b72-9a71fd65d3f8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:43.695116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aac6786e-5864-4792-8b72-9a71fd65d3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:43.803468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 665/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:44.085569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d834a6-7a9a-435d-9e61-d5722626e3c6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:44.087147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-79d834a6-7a9a-435d-9e61-d5722626e3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:44.196805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d834a6-7a9a-435d-9e61-d5722626e3c6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:44.272577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:44.273193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:44.273473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:44.273744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:44.275371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:44.292547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:44.386002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:44.387359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027a1ee4-96c7-4ab5-8a2b-1252c1408ddf service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:44.405737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 663/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:46.654823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c516-855b-4a05-89c6-3f7088ac2caf None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:46.655317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c516-855b-4a05-89c6-3f7088ac2caf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:46.655432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c516-855b-4a05-89c6-3f7088ac2caf None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:46.657158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 666/1329] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:46.663109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dde4b-5134-4713-a2f0-547075158b1f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:46.663547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dde4b-5134-4713-a2f0-547075158b1f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:46.663673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dde4b-5134-4713-a2f0-547075158b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:46.731640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-721dde4b-5134-4713-a2f0-547075158b1f None None] MFA Rules not processed for user `2921a61a62e3462084377c936b42cb02`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:46.732868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-721dde4b-5134-4713-a2f0-547075158b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:46.798414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:46.829982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 664/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:47.124016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c516-855b-4a05-89c6-3f7088ac2caf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:47.125650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4040c516-855b-4a05-89c6-3f7088ac2caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:47.236081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c516-855b-4a05-89c6-3f7088ac2caf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:47.257381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:47.314207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:47.314822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:47.315098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:47.315364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:47.317005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:47.426452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:47.428719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73accb4-f5f3-47ca-9b02-c11e4d0ef0f8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:47.446657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 667/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:49.855403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826c19-7711-4bea-92aa-2035997e0911 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:49.855907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826c19-7711-4bea-92aa-2035997e0911 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:49.855907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826c19-7711-4bea-92aa-2035997e0911 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:49.857276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 665/1332] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:49.862569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ab91a-fce8-4bfc-b056-f050b943fa75 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:49.862945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ab91a-fce8-4bfc-b056-f050b943fa75 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:49.863126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ab91a-fce8-4bfc-b056-f050b943fa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:49.925975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:49.929571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9a5ab91a-fce8-4bfc-b056-f050b943fa75 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:49.931166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9a5ab91a-fce8-4bfc-b056-f050b943fa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:50.026145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 668/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:50.307462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826c19-7711-4bea-92aa-2035997e0911 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:50.309018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-06826c19-7711-4bea-92aa-2035997e0911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:50.409747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826c19-7711-4bea-92aa-2035997e0911 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:50.483745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:50.484362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:50.484637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:50.484902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:50.486469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:50.512382 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:50.592890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:50.594237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7f6b7-8706-4969-b520-1fa79a6e2815 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:50.611897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 666/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:52.836175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b9f4-2c89-4bcc-be9c-8be4c888a63e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:52.836716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b9f4-2c89-4bcc-be9c-8be4c888a63e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:52.836716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3b9f4-2c89-4bcc-be9c-8be4c888a63e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:52.838034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 669/1335] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:52.843332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b15ce8-3914-4152-b55e-0070ec5c918a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:52.843765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b15ce8-3914-4152-b55e-0070ec5c918a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:52.843944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b15ce8-3914-4152-b55e-0070ec5c918a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:52.909423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e3b15ce8-3914-4152-b55e-0070ec5c918a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:52.910610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3b15ce8-3914-4152-b55e-0070ec5c918a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:52.984948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:53.007220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 667/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:53.299702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3b9f4-2c89-4bcc-be9c-8be4c888a63e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:53.301262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1cc3b9f4-2c89-4bcc-be9c-8be4c888a63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:53.405007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3b9f4-2c89-4bcc-be9c-8be4c888a63e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:53.435376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:53.480698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:53.481300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:53.481583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:53.481857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:53.483463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:53.591199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:53.592661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f9309-531e-499e-9ccb-9eb16967e431 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:53.610765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 670/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:50:56.112468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f46e8-87b4-49f2-bd0e-d9ab4043eae3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:56.112997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f46e8-87b4-49f2-bd0e-d9ab4043eae3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:56.113092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f46e8-87b4-49f2-bd0e-d9ab4043eae3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:56.114361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 668/1338] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:50:56.119739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97019-9121-4b67-83b1-4f3d2f49a7a4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:56.120176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97019-9121-4b67-83b1-4f3d2f49a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:56.120303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c97019-9121-4b67-83b1-4f3d2f49a7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:56.186556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a1c97019-9121-4b67-83b1-4f3d2f49a7a4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:50:56.187759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a1c97019-9121-4b67-83b1-4f3d2f49a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:56.196192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:56.283884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 671/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:50:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:50:56.575613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f46e8-87b4-49f2-bd0e-d9ab4043eae3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:50:56.577182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2a1f46e8-87b4-49f2-bd0e-d9ab4043eae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:56.690276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f46e8-87b4-49f2-bd0e-d9ab4043eae3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:50:56.770243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:50:56.770908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:50:56.771415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:50:56.771701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:50:56.773301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:50:56.810032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:50:56.886639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:50:56.888101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7c5f7-2c6f-46f4-b02d-c7611cabb9d3 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:50:56.908998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 669/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:50:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:01.129864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdf281-ef7a-4642-8ffb-152cd06a64ab None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:01.131029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdf281-ef7a-4642-8ffb-152cd06a64ab None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:01.131133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdf281-ef7a-4642-8ffb-152cd06a64ab None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:01.132912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 672/1341] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:01.137790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c22a08-c3c4-4fec-ad00-6599c69f1004 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:01.138203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c22a08-c3c4-4fec-ad00-6599c69f1004 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:01.138321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c22a08-c3c4-4fec-ad00-6599c69f1004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:01.219306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-48c22a08-c3c4-4fec-ad00-6599c69f1004 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:01.220557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-48c22a08-c3c4-4fec-ad00-6599c69f1004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:01.314614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:01.326282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 670/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:01.613031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdf281-ef7a-4642-8ffb-152cd06a64ab None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:01.614676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8ccdf281-ef7a-4642-8ffb-152cd06a64ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:01.720517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdf281-ef7a-4642-8ffb-152cd06a64ab None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:01.761122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:01.799139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:01.799883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:01.800170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:01.800469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:01.802607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:01.916406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:01.917761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4635d2-e599-4207-954c-62c9e6f3e520 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:01.936020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 673/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:04.762831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ad091-65bc-49b1-a372-aa980b45a644 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:04.763411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ad091-65bc-49b1-a372-aa980b45a644 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:04.763411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ad091-65bc-49b1-a372-aa980b45a644 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:04.765053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 671/1344] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:04.770099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7cdc0-9256-4fe0-958c-103d83ea53b9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:04.770543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7cdc0-9256-4fe0-958c-103d83ea53b9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:04.770719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7cdc0-9256-4fe0-958c-103d83ea53b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:04.840762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-85c7cdc0-9256-4fe0-958c-103d83ea53b9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:04.841981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85c7cdc0-9256-4fe0-958c-103d83ea53b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:04.860505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:04.942599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 674/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:05.232630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ad091-65bc-49b1-a372-aa980b45a644 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:05.234249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bc7ad091-65bc-49b1-a372-aa980b45a644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:05.347191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ad091-65bc-49b1-a372-aa980b45a644 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:05.421004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:05.421610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:05.421890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:05.422162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:05.423746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:05.468582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:05.537198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:05.538627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4e1b2-95ae-4637-a1cc-d327acc3bb84 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:05.556951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 672/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:09.779917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc3394-96ea-4c69-8257-6481290fb544 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:09.780503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc3394-96ea-4c69-8257-6481290fb544 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:09.780503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc3394-96ea-4c69-8257-6481290fb544 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:09.782126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 675/1347] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:09.787817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de42a1-28cd-474d-87b3-7e626bcda105 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:09.788343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de42a1-28cd-474d-87b3-7e626bcda105 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:09.788444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de42a1-28cd-474d-87b3-7e626bcda105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:09.861255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c5de42a1-28cd-474d-87b3-7e626bcda105 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:09.862590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c5de42a1-28cd-474d-87b3-7e626bcda105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:09.964939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:09.967252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 673/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:10.252322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc3394-96ea-4c69-8257-6481290fb544 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:10.253869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9dbc3394-96ea-4c69-8257-6481290fb544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:10.358927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc3394-96ea-4c69-8257-6481290fb544 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:10.406365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:10.433274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:10.433876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:10.434156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:10.434427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:10.436007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:10.547033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:10.548414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7891c2-320e-47ba-9272-9bb497f56289 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:10.566487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 676/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:15.633276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a9877-fe35-4e1c-94e3-f9912875a4e7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:15.633784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a9877-fe35-4e1c-94e3-f9912875a4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:15.633898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a9877-fe35-4e1c-94e3-f9912875a4e7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:15.635352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 674/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:51:15.640651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719ff8f-5f45-4ee0-85e3-3bf255abb5be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:15.641102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719ff8f-5f45-4ee0-85e3-3bf255abb5be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:15.641196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719ff8f-5f45-4ee0-85e3-3bf255abb5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:15.793707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c719ff8f-5f45-4ee0-85e3-3bf255abb5be None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:15.795847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c719ff8f-5f45-4ee0-85e3-3bf255abb5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:15.852638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:15.952150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 677/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:20.103535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678cfa5-5359-4428-945f-6bc5e37b70d4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:20.104022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678cfa5-5359-4428-945f-6bc5e37b70d4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:20.104188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678cfa5-5359-4428-945f-6bc5e37b70d4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:20.105547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 675/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:20.110741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536de60-362a-480d-b8b9-ee2cc3f4827d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:20.111162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536de60-362a-480d-b8b9-ee2cc3f4827d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:20.111353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536de60-362a-480d-b8b9-ee2cc3f4827d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:20.195400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f536de60-362a-480d-b8b9-ee2cc3f4827d None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:20.196643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f536de60-362a-480d-b8b9-ee2cc3f4827d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:20.272757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:20.312672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 678/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:51:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:21.758641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f45a38-036a-4111-98d4-e32ac0df760e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:21.759110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f45a38-036a-4111-98d4-e32ac0df760e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:21.759207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f45a38-036a-4111-98d4-e32ac0df760e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:21.760567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 676/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:21.766015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022aef-9db3-415f-a149-5cfe35691fb5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:21.766443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022aef-9db3-415f-a149-5cfe35691fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:21.766699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d022aef-9db3-415f-a149-5cfe35691fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:21.850012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0d022aef-9db3-415f-a149-5cfe35691fb5 None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:21.851157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d022aef-9db3-415f-a149-5cfe35691fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:21.958379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 679/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:25.526831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105613-718d-4154-942e-fc25e6f94f95 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:25.527788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105613-718d-4154-942e-fc25e6f94f95 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:25.528088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105613-718d-4154-942e-fc25e6f94f95 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:25.529441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 677/1356] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:25.534714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860bc1-3168-44f6-af1d-064316d62802 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:25.535123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860bc1-3168-44f6-af1d-064316d62802 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:25.535315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26860bc1-3168-44f6-af1d-064316d62802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:25.573116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:25.608001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-26860bc1-3168-44f6-af1d-064316d62802 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:25.609389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-26860bc1-3168-44f6-af1d-064316d62802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:25.718086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 680/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:26.034420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a28f-fd08-47a0-9575-1f5ae3743e80 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:26.036877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a28f-fd08-47a0-9575-1f5ae3743e80 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:26.037708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a28f-fd08-47a0-9575-1f5ae3743e80 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:26.042169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 678/1358] 10.222.0.57 () {54 vars in 962 bytes} [Thu Sep 15 12:51:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:26.059822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665d58b-71ad-404f-ad35-01104735a88a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:26.060463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665d58b-71ad-404f-ad35-01104735a88a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:26.061309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665d58b-71ad-404f-ad35-01104735a88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:26.150653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7665d58b-71ad-404f-ad35-01104735a88a None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:26.152051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7665d58b-71ad-404f-ad35-01104735a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:26.212701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:26.288255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 681/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Sep 15 12:51:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:26.296216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a28f-fd08-47a0-9575-1f5ae3743e80 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:26.297732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ed12a28f-fd08-47a0-9575-1f5ae3743e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:26.452315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a28f-fd08-47a0-9575-1f5ae3743e80 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:26.530720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:26.579350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:26.579869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:26.580144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:26.580448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:26.582053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:26.680088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:26.681746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b906635-5aef-478a-a0f4-f41d02f3941d service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:26.702652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 679/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:29.158430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8058-59ff-4d7a-83ea-c6f45200998d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:29.159086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8058-59ff-4d7a-83ea-c6f45200998d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:29.159086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8058-59ff-4d7a-83ea-c6f45200998d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:29.166985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa29bf8-4186-4183-97c4-7f33130b93ff None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:29.166985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa29bf8-4186-4183-97c4-7f33130b93ff None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:29.167760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa29bf8-4186-4183-97c4-7f33130b93ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:29.172202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 682/1361] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:29 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:29.235521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cfa29bf8-4186-4183-97c4-7f33130b93ff None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:29.236747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cfa29bf8-4186-4183-97c4-7f33130b93ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:29.282273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:29.350075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 680/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:29.637911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8058-59ff-4d7a-83ea-c6f45200998d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:29.639425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c2b8058-59ff-4d7a-83ea-c6f45200998d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:29.826948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8058-59ff-4d7a-83ea-c6f45200998d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:29.840056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:29.962648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:29.963249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:29.963534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:29.963804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:29.965443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:30.058568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:30.060088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c99c5d-c5f7-4e9d-bb6f-dcb7a86bfbba service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:30.079289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 683/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:32.612292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3af975-c918-4c15-9804-b344122a0794 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:32.612828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3af975-c918-4c15-9804-b344122a0794 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:32.612828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3af975-c918-4c15-9804-b344122a0794 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:32.615402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 681/1364] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:32.620525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a797-e6bc-4f7f-a915-ed50268b105a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:32.620956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a797-e6bc-4f7f-a915-ed50268b105a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:32.621150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56a797-e6bc-4f7f-a915-ed50268b105a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:32.637117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:32.693128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fb56a797-e6bc-4f7f-a915-ed50268b105a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:32.694431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb56a797-e6bc-4f7f-a915-ed50268b105a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:32.801422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 684/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:33.103835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8ba72-c744-4ff8-bc55-df97f7ee45d5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:33.104204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8ba72-c744-4ff8-bc55-df97f7ee45d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:33.104625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8ba72-c744-4ff8-bc55-df97f7ee45d5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:33.106334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 682/1366] 10.222.0.57 () {54 vars in 962 bytes} [Thu Sep 15 12:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:33.111633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdae8d-817f-4154-b892-95514f1e71aa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:33.111987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdae8d-817f-4154-b892-95514f1e71aa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:33.112355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdae8d-817f-4154-b892-95514f1e71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:33.193783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-effdae8d-817f-4154-b892-95514f1e71aa None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:33.195011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-effdae8d-817f-4154-b892-95514f1e71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:33.233485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:33.322044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 685/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Sep 15 12:51:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:33.329705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8ba72-c744-4ff8-bc55-df97f7ee45d5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:33.331219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41f8ba72-c744-4ff8-bc55-df97f7ee45d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:33.487677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8ba72-c744-4ff8-bc55-df97f7ee45d5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:33.509908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:33.610739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:33.611357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:33.611643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:33.611913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:33.614133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:33.713866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:33.715485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f30b4-eb7e-47db-8ee6-fdafb2b1073f service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:33.737292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 683/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:33.851742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aafc20-2577-41d1-b61b-0808dddf7147 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:33.852381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aafc20-2577-41d1-b61b-0808dddf7147 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:33.852381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aafc20-2577-41d1-b61b-0808dddf7147 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:33.856355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 686/1369] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:33.859878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cef7e-3e6c-4228-be0c-d9658514da74 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:33.860170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cef7e-3e6c-4228-be0c-d9658514da74 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:33.860523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cef7e-3e6c-4228-be0c-d9658514da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:33.899437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:33.944890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ff9cef7e-3e6c-4228-be0c-d9658514da74 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:33.946192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ff9cef7e-3e6c-4228-be0c-d9658514da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:34.075248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 684/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:34.428313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39f4ec-66ea-411a-ae70-cf96c22372a6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:34.428778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39f4ec-66ea-411a-ae70-cf96c22372a6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:34.429110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39f4ec-66ea-411a-ae70-cf96c22372a6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:34.430815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 687/1371] 10.222.0.57 () {54 vars in 964 bytes} [Thu Sep 15 12:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:34.435601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb966b-7ce0-425e-9ce9-67937dcb9aa9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:34.435994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb966b-7ce0-425e-9ce9-67937dcb9aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:34.436409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb966b-7ce0-425e-9ce9-67937dcb9aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:34.517127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-09eb966b-7ce0-425e-9ce9-67937dcb9aa9 None None] MFA Rules not processed for user `8d2458e0ed4b4fceb437ac5c7b43d598`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:34.518486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-09eb966b-7ce0-425e-9ce9-67937dcb9aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:34.569287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:34.601560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 685/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Thu Sep 15 12:51:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:34.609151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39f4ec-66ea-411a-ae70-cf96c22372a6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:34.610779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e39f4ec-66ea-411a-ae70-cf96c22372a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:34.722294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39f4ec-66ea-411a-ae70-cf96c22372a6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:34.769183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:34.813720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:34.814498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:34.814700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:34.814998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:34.816939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:34.960459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:34.961708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fb2b7-614d-409b-8928-c3ae4f0ed7b1 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:34.981676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 688/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:36.015662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956beeaf-1085-4b56-ad52-94d6e5d8a7ef None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:36.016230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956beeaf-1085-4b56-ad52-94d6e5d8a7ef None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:36.016230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956beeaf-1085-4b56-ad52-94d6e5d8a7ef None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:36.017576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 686/1374] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:51:36.023920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac523f-619b-4ee4-b270-0df4a71e9a49 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:36.024406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac523f-619b-4ee4-b270-0df4a71e9a49 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:36.024578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac523f-619b-4ee4-b270-0df4a71e9a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:36.095068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ceac523f-619b-4ee4-b270-0df4a71e9a49 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:36.095068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ceac523f-619b-4ee4-b270-0df4a71e9a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:36.116899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:36.197202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 689/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:36.502601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956beeaf-1085-4b56-ad52-94d6e5d8a7ef None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:36.504212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-956beeaf-1085-4b56-ad52-94d6e5d8a7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:36.666527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956beeaf-1085-4b56-ad52-94d6e5d8a7ef None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:36.721381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:36.789893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:36.790503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:36.790781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:36.791044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:36.792710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:36.891686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:36.893329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6cbbc-0ca6-49fc-ab8c-cf8fe74622e7 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:36.912809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 687/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:39.186096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385883a-4190-46a8-af09-e4034d731579 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:39.186642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385883a-4190-46a8-af09-e4034d731579 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:39.186904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385883a-4190-46a8-af09-e4034d731579 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:39.188557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 690/1377] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:39.194097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd774d4-d0d0-4352-8a75-32e2edca8c48 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:39.194519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd774d4-d0d0-4352-8a75-32e2edca8c48 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:39.194834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd774d4-d0d0-4352-8a75-32e2edca8c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:39.264511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2bd774d4-d0d0-4352-8a75-32e2edca8c48 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:39.266001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2bd774d4-d0d0-4352-8a75-32e2edca8c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:39.275903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:39.377582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 688/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:39.673187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385883a-4190-46a8-af09-e4034d731579 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:39.674884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8385883a-4190-46a8-af09-e4034d731579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:39.833212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385883a-4190-46a8-af09-e4034d731579 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:39.837330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:39.953468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:39.954252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:39.954639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:39.955020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:39.956779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:40.054419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:40.056324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4080bc-2c08-49b8-8a64-6565ad5d86db service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:40.077605 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 691/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:42.343220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0757ba-8eb2-4351-a4fe-44a8fac52b51 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:42.343686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0757ba-8eb2-4351-a4fe-44a8fac52b51 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:42.343788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0757ba-8eb2-4351-a4fe-44a8fac52b51 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:42.345271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 689/1380] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:42.350772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f0be-e659-4526-8b7d-743edd9bd9df None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:42.351211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f0be-e659-4526-8b7d-743edd9bd9df None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:42.351313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f0be-e659-4526-8b7d-743edd9bd9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:42.363938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:42.430080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-26f5f0be-e659-4526-8b7d-743edd9bd9df None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:42.431419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-26f5f0be-e659-4526-8b7d-743edd9bd9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:42.536924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 692/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:42.880934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0757ba-8eb2-4351-a4fe-44a8fac52b51 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:42.882452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bd0757ba-8eb2-4351-a4fe-44a8fac52b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:43.049635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:43.061167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0757ba-8eb2-4351-a4fe-44a8fac52b51 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:43.179959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:43.180513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:43.180790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:43.181059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:43.182647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:43.279438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:43.281090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1455f-147b-407f-b3d1-8042fd23a1cc service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:43.291900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:43.301314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 690/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:45.525887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ee93a-8c9b-4ecd-99d1-0aa47195add2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:45.526580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ee93a-8c9b-4ecd-99d1-0aa47195add2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:45.526580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ee93a-8c9b-4ecd-99d1-0aa47195add2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:45.527940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 693/1383] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:45.533091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54185-0573-4826-b570-9a302a6a9ed0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:45.533598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54185-0573-4826-b570-9a302a6a9ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:45.533770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54185-0573-4826-b570-9a302a6a9ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:45.602887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-09a54185-0573-4826-b570-9a302a6a9ed0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:45.604107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-09a54185-0573-4826-b570-9a302a6a9ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:45.712861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 691/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:46.001646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ee93a-8c9b-4ecd-99d1-0aa47195add2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:46.003167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-620ee93a-8c9b-4ecd-99d1-0aa47195add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:46.107246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:46.171721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ee93a-8c9b-4ecd-99d1-0aa47195add2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:46.287947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:46.288575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:46.288854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:46.289127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:46.290755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:46.343603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:46.387058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:46.388707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf967f-e0ef-40c1-87fa-bfc1e06b6d69 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:46.407370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 694/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:48.936429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66a865-2abb-4062-bc48-6daf082c4a6c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:48.937093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66a865-2abb-4062-bc48-6daf082c4a6c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:48.937093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66a865-2abb-4062-bc48-6daf082c4a6c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:48.938312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 692/1386] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:48.943525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de862b02-cf15-4eed-8136-0336528de2a5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:48.943974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de862b02-cf15-4eed-8136-0336528de2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:48.944078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de862b02-cf15-4eed-8136-0336528de2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:49.014387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-de862b02-cf15-4eed-8136-0336528de2a5 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:49.015852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de862b02-cf15-4eed-8136-0336528de2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:49.118043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 695/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:49.417194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66a865-2abb-4062-bc48-6daf082c4a6c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:49.418712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cd66a865-2abb-4062-bc48-6daf082c4a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:49.497415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:49.567482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66a865-2abb-4062-bc48-6daf082c4a6c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:49.682917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:49.683476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:49.683758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:49.684032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:49.685652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:49.725538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:49.778282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:49.779883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a0225-044f-42bf-83f8-a3d282981108 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:49.798693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 693/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:50.258863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77c81-7ecb-4593-bbf2-cd22e0c3537b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:50.259441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77c81-7ecb-4593-bbf2-cd22e0c3537b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:50.259534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77c81-7ecb-4593-bbf2-cd22e0c3537b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:50.261004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 696/1389] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:50.266742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29401c2f-0044-4ba1-befa-0698aeb7fee8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:50.267136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29401c2f-0044-4ba1-befa-0698aeb7fee8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:50.267375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29401c2f-0044-4ba1-befa-0698aeb7fee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:50.336248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-29401c2f-0044-4ba1-befa-0698aeb7fee8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:50.337465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-29401c2f-0044-4ba1-befa-0698aeb7fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:50.433793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 694/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:50.737589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9267f17-c625-41ef-a656-7686f7bf3d24 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:50.737897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9267f17-c625-41ef-a656-7686f7bf3d24 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:50.738172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9267f17-c625-41ef-a656-7686f7bf3d24 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:50.739507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 697/1391] 10.222.0.57 () {54 vars in 962 bytes} [Thu Sep 15 12:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:50.744881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d1fe8-527c-42f9-a92b-33f63e435b73 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:50.745240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d1fe8-527c-42f9-a92b-33f63e435b73 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:50.745590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d1fe8-527c-42f9-a92b-33f63e435b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:50.762169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:50.828265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-556d1fe8-527c-42f9-a92b-33f63e435b73 None None] MFA Rules not processed for user `6e29455534ee474899b4f470d2a55f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:50.829548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-556d1fe8-527c-42f9-a92b-33f63e435b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:50.916711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 695/1392] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Sep 15 12:51:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:50.926412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9267f17-c625-41ef-a656-7686f7bf3d24 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:50.928200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9267f17-c625-41ef-a656-7686f7bf3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:50.963109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:51.041997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9267f17-c625-41ef-a656-7686f7bf3d24 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:51.121433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:51.122047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:51.122329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:51.122602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:51.124196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:51.203239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:51.237728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:51.239833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c5d387-7714-4db7-9467-f6cb87d0a2d9 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:51.258612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 698/1393] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:51:52.005841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13df57-013e-430b-bedd-2192f54b51a9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.006359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13df57-013e-430b-bedd-2192f54b51a9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.006475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13df57-013e-430b-bedd-2192f54b51a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.012569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343f3b-5a47-4568-8fb8-21e7982c711c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.013029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343f3b-5a47-4568-8fb8-21e7982c711c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.013318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343f3b-5a47-4568-8fb8-21e7982c711c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.014749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 699/1394] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:52.020908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94187b-204a-4ccc-9e7f-9378409c97a3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.021191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94187b-204a-4ccc-9e7f-9378409c97a3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.021466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94187b-204a-4ccc-9e7f-9378409c97a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.078294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9c13df57-013e-430b-bedd-2192f54b51a9 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:52.079522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9c13df57-013e-430b-bedd-2192f54b51a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.094244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dd94187b-204a-4ccc-9e7f-9378409c97a3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:52.096342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dd94187b-204a-4ccc-9e7f-9378409c97a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.139469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:52.172309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 696/1395] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:52.198626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 700/1396] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:51:52.216968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda30267-4b96-47a1-ad13-85834db1e318 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.217258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda30267-4b96-47a1-ad13-85834db1e318 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.217541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda30267-4b96-47a1-ad13-85834db1e318 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.218887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 697/1397] 10.222.0.57 () {54 vars in 943 bytes} [Thu Sep 15 12:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:52.225904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.226263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.226602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.273394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:52.305905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] MFA Rules not processed for user `4ead9001bd3241399dac6579a9588975`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:52.307112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.416888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 701/1398] 10.222.0.57 () {58 vars in 1040 bytes} [Thu Sep 15 12:51:52 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:52.426137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda30267-4b96-47a1-ad13-85834db1e318 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:52.427645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fda30267-4b96-47a1-ad13-85834db1e318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.488483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:52.489836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.573998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda30267-4b96-47a1-ad13-85834db1e318 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:52.596676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:52.600038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c4867-92fc-4359-971a-ab9d0009f644 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:52.635281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:52.683940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:52.684576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.684846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.685115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.686729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.720558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:52.721174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:52.721456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:52.721730 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:52.723403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:52.773460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:52.774973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997087-a81f-4cb1-bdea-240101ef449d service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:52.783098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 698/1399] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 12:51:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:51:52.818275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:52.819933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be65a75-2c21-47b9-8c39-e3de750fb824 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:52.839173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 702/1400] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:51:55.056328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b8605-7fda-4588-afd7-08ac121c0e5d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:55.056913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b8605-7fda-4588-afd7-08ac121c0e5d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:55.057017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b8605-7fda-4588-afd7-08ac121c0e5d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:55.058559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 699/1401] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:55.063647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c57a6b-5946-453b-b608-825008e43195 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:55.064053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c57a6b-5946-453b-b608-825008e43195 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:55.064243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c57a6b-5946-453b-b608-825008e43195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:55.070201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:55.145067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81c57a6b-5946-453b-b608-825008e43195 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:55.146313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81c57a6b-5946-453b-b608-825008e43195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:55.259463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 703/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:55.550816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b8605-7fda-4588-afd7-08ac121c0e5d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:55.552413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4d8b8605-7fda-4588-afd7-08ac121c0e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:55.656103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:55.722140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b8605-7fda-4588-afd7-08ac121c0e5d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:55.850332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:55.851043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:55.851326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:55.851600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:55.853236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:55.958997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:56.035997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:56.035997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab6f60-fe2c-461f-9288-82e56d65f5ea service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:56.072983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 700/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:51:58.333588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162e02-8923-4b7d-8ff4-834dc247ac3c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:58.334150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162e02-8923-4b7d-8ff4-834dc247ac3c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:58.334150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f162e02-8923-4b7d-8ff4-834dc247ac3c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:58.335487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 704/1404] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:51:58.342404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2cc86-2329-43b5-bb47-6d6f729e9905 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:58.342879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2cc86-2329-43b5-bb47-6d6f729e9905 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:58.342970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2cc86-2329-43b5-bb47-6d6f729e9905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:58.413749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-afc2cc86-2329-43b5-bb47-6d6f729e9905 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:51:58.414975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-afc2cc86-2329-43b5-bb47-6d6f729e9905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:58.528426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 701/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:51:58.822208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162e02-8923-4b7d-8ff4-834dc247ac3c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:51:58.824011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6f162e02-8923-4b7d-8ff4-834dc247ac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:58.919208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:58.995903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f162e02-8923-4b7d-8ff4-834dc247ac3c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:51:59.137397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:51:59.138086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:51:59.138373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:51:59.138641 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:51:59.140525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:51:59.193908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:51:59.240053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:51:59.241936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c191a-3108-4946-8f76-46aaaf46a616 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:51:59.262311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 705/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:01.504925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23093c-4403-4497-8179-bb6d6776489f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:01.505507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23093c-4403-4497-8179-bb6d6776489f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:01.505507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23093c-4403-4497-8179-bb6d6776489f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:01.506897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 702/1407] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:01.512358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314d7b-9bba-4a42-8770-549452e28a9a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:01.512772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314d7b-9bba-4a42-8770-549452e28a9a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:01.512924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314d7b-9bba-4a42-8770-549452e28a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:01.586650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4a314d7b-9bba-4a42-8770-549452e28a9a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:01.587882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a314d7b-9bba-4a42-8770-549452e28a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:01.697918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 706/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:01.994616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23093c-4403-4497-8179-bb6d6776489f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:01.998475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7d23093c-4403-4497-8179-bb6d6776489f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:02.042003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:02.174001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23093c-4403-4497-8179-bb6d6776489f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:02.301239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:02.307340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:02.307958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:02.308240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:02.308592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:02.310201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:02.412486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:02.414091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27ea66-03e9-4863-a231-0b40fb87f89b service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:02.440539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 703/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:11.696261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:12.430357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=47 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:15.812659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57490f16-eaca-4482-a275-3cb41dc01056 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:15.813214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57490f16-eaca-4482-a275-3cb41dc01056 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:15.813214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57490f16-eaca-4482-a275-3cb41dc01056 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:15.814569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 707/1410] 10.222.0.57 () {54 vars in 918 bytes} [Thu Sep 15 12:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:15.819731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01284874-6bb0-4217-a0fa-904ecfa2be42 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:15.820145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01284874-6bb0-4217-a0fa-904ecfa2be42 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:15.820294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01284874-6bb0-4217-a0fa-904ecfa2be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:15.896999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-01284874-6bb0-4217-a0fa-904ecfa2be42 None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:15.898111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-01284874-6bb0-4217-a0fa-904ecfa2be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:16.005356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 704/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Sep 15 12:52:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:16.020684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d62e-dd64-4e5f-a296-81712ed77656 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:16.020974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d62e-dd64-4e5f-a296-81712ed77656 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:16.021253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015d62e-dd64-4e5f-a296-81712ed77656 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:16.022610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 708/1412] 10.222.0.57 () {54 vars in 918 bytes} [Thu Sep 15 12:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:16.027352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8118-147d-411b-a899-223287e461bb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:16.027638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8118-147d-411b-a899-223287e461bb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:16.027912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8118-147d-411b-a899-223287e461bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:16.102204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:16.105531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-96da8118-147d-411b-a899-223287e461bb None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:16.107263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96da8118-147d-411b-a899-223287e461bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:16.221978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 705/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Sep 15 12:52:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:18.828647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb71ce-fa8e-40c0-8e7c-95c5b6a136a3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:18.829217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb71ce-fa8e-40c0-8e7c-95c5b6a136a3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:18.829217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb71ce-fa8e-40c0-8e7c-95c5b6a136a3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:18.830535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:18.836650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce6326-fa45-42f4-8019-062302955266 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:18.837080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce6326-fa45-42f4-8019-062302955266 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:18.837177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce6326-fa45-42f4-8019-062302955266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:18.903057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a5ce6326-fa45-42f4-8019-062302955266 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:18.904263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a5ce6326-fa45-42f4-8019-062302955266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:18.961326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:19.002867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:19.312690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb71ce-fa8e-40c0-8e7c-95c5b6a136a3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:19.314351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d1eb71ce-fa8e-40c0-8e7c-95c5b6a136a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:19.459767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb71ce-fa8e-40c0-8e7c-95c5b6a136a3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:19.533243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:19.575113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:19.575728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:19.576012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:19.576314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:19.577935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:19.670420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:19.672009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288cb28-6384-475d-940c-ed4a1980ac99 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:19.690367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 710/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:20.835825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828343d2-146e-4f66-9e57-51ba989ed45a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:20.836250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828343d2-146e-4f66-9e57-51ba989ed45a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:20.836401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828343d2-146e-4f66-9e57-51ba989ed45a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:20.837712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:20.844045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf884dc-33b3-44a6-9b97-af40bc4e1b87 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:20.844480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf884dc-33b3-44a6-9b97-af40bc4e1b87 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:20.844635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf884dc-33b3-44a6-9b97-af40bc4e1b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:20.920261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-acf884dc-33b3-44a6-9b97-af40bc4e1b87 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:20.921514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-acf884dc-33b3-44a6-9b97-af40bc4e1b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:20.949050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:21.020385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:21.171835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828343d2-146e-4f66-9e57-51ba989ed45a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:21.173385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-828343d2-146e-4f66-9e57-51ba989ed45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:21.318412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828343d2-146e-4f66-9e57-51ba989ed45a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:21.358374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:21.431490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:21.432096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:21.432406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:21.432681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:21.434299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:21.523180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:21.524845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a196f8-db28-4bda-bef9-17cc206d5997 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:21.543164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 708/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:22.678794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baad9dd-281a-471e-b204-87c303fdf110 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:22.679325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baad9dd-281a-471e-b204-87c303fdf110 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:22.679420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baad9dd-281a-471e-b204-87c303fdf110 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:22.680757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:22.686896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14b6e-b2e7-423a-a1c3-c29342b4d189 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:22.687298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14b6e-b2e7-423a-a1c3-c29342b4d189 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:22.687457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c14b6e-b2e7-423a-a1c3-c29342b4d189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:22.742904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:22.766814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c9c14b6e-b2e7-423a-a1c3-c29342b4d189 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:22.768037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c9c14b6e-b2e7-423a-a1c3-c29342b4d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:22.865183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:23.017980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baad9dd-281a-471e-b204-87c303fdf110 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:23.019725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7baad9dd-281a-471e-b204-87c303fdf110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:23.165201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baad9dd-281a-471e-b204-87c303fdf110 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:23.177855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:23.279934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:23.280544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:23.280829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:23.281106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:23.282709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:23.371708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:23.373322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888f489-1a68-401f-994a-bc8684a0c34c service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:23.392029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 713/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:24.514575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98119dda-8856-4153-bc06-5b8385966312 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:24.515391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98119dda-8856-4153-bc06-5b8385966312 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:24.515391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98119dda-8856-4153-bc06-5b8385966312 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:24.516724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:52:24.522980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc2154-aacc-4914-b951-cc8dff1b96a1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:24.523430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc2154-aacc-4914-b951-cc8dff1b96a1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:24.523539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc2154-aacc-4914-b951-cc8dff1b96a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:24.543667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:24.601578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fecc2154-aacc-4914-b951-cc8dff1b96a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:24.602718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fecc2154-aacc-4914-b951-cc8dff1b96a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:24.700007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:24.851685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98119dda-8856-4153-bc06-5b8385966312 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:24.853150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-98119dda-8856-4153-bc06-5b8385966312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:24.961708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:24.997049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98119dda-8856-4153-bc06-5b8385966312 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:25.108061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:25.108662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:25.108938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:25.109205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:25.110786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:25.186488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:25.200240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:25.201869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3727f-0886-47d7-9340-37fd612ff190 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:25.219983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 711/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:26.339147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c0e97-eed5-4420-8d10-6d868b6cb3ad None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:26.339665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c0e97-eed5-4420-8d10-6d868b6cb3ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:26.339665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c0e97-eed5-4420-8d10-6d868b6cb3ad None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:26.341070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:26.348396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0554e-3414-42cd-812b-bc1e90d7d07c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:26.348876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0554e-3414-42cd-812b-bc1e90d7d07c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:26.348974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0554e-3414-42cd-812b-bc1e90d7d07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:26.425001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bbc0554e-3414-42cd-812b-bc1e90d7d07c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:26.426476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbc0554e-3414-42cd-812b-bc1e90d7d07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:26.522112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:26.672323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c0e97-eed5-4420-8d10-6d868b6cb3ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:26.673771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c2c0e97-eed5-4420-8d10-6d868b6cb3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:26.769880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:26.831995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c0e97-eed5-4420-8d10-6d868b6cb3ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:26.951814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:26.952385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:26.952669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:26.952939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:26.954553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:27.004184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:27.045655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:27.047253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bfdfa-ef2f-4308-b47d-6107dead7f8e service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:27.065461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 716/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:28.191519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779212-0715-46cd-a3de-46db3fffaecb None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:28.192083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779212-0715-46cd-a3de-46db3fffaecb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:28.192083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3779212-0715-46cd-a3de-46db3fffaecb None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:28.193358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:28.200680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc856c47-4b95-4972-be47-286b05efb397 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:28.201163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc856c47-4b95-4972-be47-286b05efb397 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:28.201282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc856c47-4b95-4972-be47-286b05efb397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:28.293312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bc856c47-4b95-4972-be47-286b05efb397 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:28.294531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bc856c47-4b95-4972-be47-286b05efb397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:28.397949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:28.581034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779212-0715-46cd-a3de-46db3fffaecb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:28.582955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3779212-0715-46cd-a3de-46db3fffaecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:28.635710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:28.736362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3779212-0715-46cd-a3de-46db3fffaecb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:28.850830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:28.851343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:28.851623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:28.851893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:28.853518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:28.860418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:28.945818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:28.947412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73f7b8-7814-41aa-8e0d-8e9802000c5c service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:28.965496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 714/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:30.026703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc2b3f-660a-41b7-b0ff-96ec76337494 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:30.027291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc2b3f-660a-41b7-b0ff-96ec76337494 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:30.027291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc2b3f-660a-41b7-b0ff-96ec76337494 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:30.028657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:30.035460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab96548-f11e-440e-b014-fd533eabe2c1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:30.035934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab96548-f11e-440e-b014-fd533eabe2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:30.036050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab96548-f11e-440e-b014-fd533eabe2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:30.111710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3ab96548-f11e-440e-b014-fd533eabe2c1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:30.113809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ab96548-f11e-440e-b014-fd533eabe2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:30.180572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:30.209290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:30.357835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc2b3f-660a-41b7-b0ff-96ec76337494 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:30.359271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-05dc2b3f-660a-41b7-b0ff-96ec76337494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:30.377128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:30.505661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc2b3f-660a-41b7-b0ff-96ec76337494 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:30.593231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:30.620339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:30.620895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:30.621176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:30.621449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:30.623055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:30.711936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:30.713595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551b3c0-1dc8-4c49-9c9f-e3d03e88534a service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:30.732076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 719/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:31.796803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f5859-8f15-454a-bb5c-476297ed0832 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:31.797334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f5859-8f15-454a-bb5c-476297ed0832 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:31.797334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f5859-8f15-454a-bb5c-476297ed0832 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:31.798640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:31.805359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff0ab-9400-47c6-bb38-ffcf96ec9ff0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:31.805811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff0ab-9400-47c6-bb38-ffcf96ec9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:31.805920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deff0ab-9400-47c6-bb38-ffcf96ec9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:31.882254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4deff0ab-9400-47c6-bb38-ffcf96ec9ff0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:31.883487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4deff0ab-9400-47c6-bb38-ffcf96ec9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:31.925699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:31.981439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:32.134274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f5859-8f15-454a-bb5c-476297ed0832 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:32.135786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc0f5859-8f15-454a-bb5c-476297ed0832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:32.280722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f5859-8f15-454a-bb5c-476297ed0832 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:32.334339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:32.392727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:32.393265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:32.393546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:32.394371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:32.396024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:32.484457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:32.486047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb1257-a721-4b53-800e-9dacabf4a7e2 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:32.504842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 717/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:33.560200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e485102-a758-4bde-8c2d-91ea825004e8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:33.560795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e485102-a758-4bde-8c2d-91ea825004e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:33.560795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e485102-a758-4bde-8c2d-91ea825004e8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:33.562139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:33.569098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89663ec5-60c9-4c6f-80c4-a9920e08e8a4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:33.569569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89663ec5-60c9-4c6f-80c4-a9920e08e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:33.569690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89663ec5-60c9-4c6f-80c4-a9920e08e8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:33.644623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-89663ec5-60c9-4c6f-80c4-a9920e08e8a4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:33.645819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-89663ec5-60c9-4c6f-80c4-a9920e08e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:33.647805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:33.741958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:33.892646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e485102-a758-4bde-8c2d-91ea825004e8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:33.894185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e485102-a758-4bde-8c2d-91ea825004e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:34.039909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e485102-a758-4bde-8c2d-91ea825004e8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:34.068865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:34.155138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:34.155680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:34.155970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:34.156225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:34.157859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:34.246450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:34.248035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed1b6d-a18c-4873-a967-e7fdb1015f70 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:34.266710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 722/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:36.352493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49e88-22e9-4c02-a731-317406ecc9e8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:36.353095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49e88-22e9-4c02-a731-317406ecc9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:36.353095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49e88-22e9-4c02-a731-317406ecc9e8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:36.354463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:36.359848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240aa50-4375-4d49-b810-e0d0b6a63260 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:36.360292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240aa50-4375-4d49-b810-e0d0b6a63260 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:36.360450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240aa50-4375-4d49-b810-e0d0b6a63260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:36.401522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:36.427751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c240aa50-4375-4d49-b810-e0d0b6a63260 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:36.429007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c240aa50-4375-4d49-b810-e0d0b6a63260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:36.525717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:36.814630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49e88-22e9-4c02-a731-317406ecc9e8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:36.816717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96e49e88-22e9-4c02-a731-317406ecc9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:36.924326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49e88-22e9-4c02-a731-317406ecc9e8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:36.957431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:36.998339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:36.998936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:36.999211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:36.999480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:37.001096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:37.111361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:37.113349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090e318-e0b7-49ec-9796-1be6eba351cf service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:37.131995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 720/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:39.452217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d449901-0032-4445-9246-0570ee684260 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:39.452785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d449901-0032-4445-9246-0570ee684260 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:39.452785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d449901-0032-4445-9246-0570ee684260 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:39.454047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:39.459930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926480ad-2660-4ae5-86bb-aa1058add2eb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:39.460362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926480ad-2660-4ae5-86bb-aa1058add2eb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:39.460510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926480ad-2660-4ae5-86bb-aa1058add2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:39.526529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-926480ad-2660-4ae5-86bb-aa1058add2eb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:39.527741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-926480ad-2660-4ae5-86bb-aa1058add2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:39.541005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:39.625577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:39.918146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d449901-0032-4445-9246-0570ee684260 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:39.919688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d449901-0032-4445-9246-0570ee684260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:40.022959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d449901-0032-4445-9246-0570ee684260 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:40.035499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:40.096519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:40.097181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:40.097969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:40.097969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:40.099299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:40.209209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:40.210635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc310b-395d-4dc8-9fc0-e95fdb86f1f0 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:40.228822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 725/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:42.703172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff9ce4-ddfd-48cf-b271-73d71b3e6281 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:42.703715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff9ce4-ddfd-48cf-b271-73d71b3e6281 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:42.703715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff9ce4-ddfd-48cf-b271-73d71b3e6281 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:42.705144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:42.710891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9906571-5091-4bd1-9c0c-f38991ff8ade None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:42.711299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9906571-5091-4bd1-9c0c-f38991ff8ade None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:42.711448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9906571-5091-4bd1-9c0c-f38991ff8ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:42.760653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:42.779102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c9906571-5091-4bd1-9c0c-f38991ff8ade None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:42.780346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c9906571-5091-4bd1-9c0c-f38991ff8ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:42.877375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:43.163404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff9ce4-ddfd-48cf-b271-73d71b3e6281 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:43.165128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4dff9ce4-ddfd-48cf-b271-73d71b3e6281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:43.270345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff9ce4-ddfd-48cf-b271-73d71b3e6281 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:43.311326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:43.344693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:43.345298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:43.345578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:43.345851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:43.348123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:43.457451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:43.458883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139465e-693d-462b-86a4-fdc8c9701e99 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:43.477314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 723/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:45.756426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e501b-7d87-485f-9475-f3f38a935b51 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:45.757002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e501b-7d87-485f-9475-f3f38a935b51 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:45.757002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e501b-7d87-485f-9475-f3f38a935b51 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:45.758404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:45.763596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aeeaf-317d-4ef7-8ec6-69832de0588c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:45.764019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aeeaf-317d-4ef7-8ec6-69832de0588c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:45.764123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aeeaf-317d-4ef7-8ec6-69832de0588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:45.830394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-391aeeaf-317d-4ef7-8ec6-69832de0588c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:45.831881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-391aeeaf-317d-4ef7-8ec6-69832de0588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:45.854290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:45.930689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:46.214805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e501b-7d87-485f-9475-f3f38a935b51 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:46.216409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f3e501b-7d87-485f-9475-f3f38a935b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:46.320085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e501b-7d87-485f-9475-f3f38a935b51 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:46.340168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:46.394050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:46.394659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:46.394944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:46.395211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:46.396832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:46.505497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:46.506858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44be06-8202-44cd-970d-23636cf43396 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:46.524849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 728/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:48.745309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6473-dfbe-477e-a398-c5566e144542 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:48.745875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6473-dfbe-477e-a398-c5566e144542 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:48.745875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6473-dfbe-477e-a398-c5566e144542 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:48.747173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:48.753206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b56f29-f76a-47d6-a2c8-a513cf57aaeb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:48.753609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b56f29-f76a-47d6-a2c8-a513cf57aaeb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:48.753874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b56f29-f76a-47d6-a2c8-a513cf57aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:48.818036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:48.820788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-39b56f29-f76a-47d6-a2c8-a513cf57aaeb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:48.822620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-39b56f29-f76a-47d6-a2c8-a513cf57aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:48.919345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:52:49.207300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a6473-dfbe-477e-a398-c5566e144542 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:49.208885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-345a6473-dfbe-477e-a398-c5566e144542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:49.311529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a6473-dfbe-477e-a398-c5566e144542 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:49.361146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:49.385856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:49.386458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:49.386737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:49.387005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:49.388617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:49.497988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:49.499426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e40b6-0ff1-45f5-9a70-472e707976db service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:49.517439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 726/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:51.731548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cebdd-7151-4003-93c1-f68887af8cbd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:51.732062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cebdd-7151-4003-93c1-f68887af8cbd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:51.732156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cebdd-7151-4003-93c1-f68887af8cbd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:51.733521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:51.739565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298625f0-fe43-4f0f-9273-a65e5f17a8a3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:51.740033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298625f0-fe43-4f0f-9273-a65e5f17a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:51.740143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298625f0-fe43-4f0f-9273-a65e5f17a8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:51.807178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-298625f0-fe43-4f0f-9273-a65e5f17a8a3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:51.808448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-298625f0-fe43-4f0f-9273-a65e5f17a8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:51.837642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:51.907653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:52.192436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cebdd-7151-4003-93c1-f68887af8cbd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:52.193961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-996cebdd-7151-4003-93c1-f68887af8cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:52.297766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cebdd-7151-4003-93c1-f68887af8cbd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:52.326472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:52.371992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:52.372626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:52.372905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:52.373175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:52.374740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:52.484570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:52.485995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01c29a-3a47-45b7-8f3b-01e9e8359258 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:52.621941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 731/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:54.845314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065660ff-73e0-40d8-b3c9-e152f937c5da None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:54.845866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065660ff-73e0-40d8-b3c9-e152f937c5da None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:54.845866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065660ff-73e0-40d8-b3c9-e152f937c5da None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:54.847757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:54.853323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dd988-299f-4146-8420-652dc896497a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:54.853745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dd988-299f-4146-8420-652dc896497a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:54.853872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dd988-299f-4146-8420-652dc896497a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:54.919714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6d9dd988-299f-4146-8420-652dc896497a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:54.920951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6d9dd988-299f-4146-8420-652dc896497a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:54.929338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:55.017999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:55.309825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065660ff-73e0-40d8-b3c9-e152f937c5da None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:55.311403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-065660ff-73e0-40d8-b3c9-e152f937c5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:55.415193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065660ff-73e0-40d8-b3c9-e152f937c5da None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:55.470068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:55.492675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:55.493281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:55.493562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:55.493838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:55.495435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:55.603586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:55.604974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac93ff-267c-4e4a-89b3-b8bcd90a9dae service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:55.622612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 729/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:57.844687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241199c2-45ea-4b46-b16c-6246cdd96deb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:57.845282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241199c2-45ea-4b46-b16c-6246cdd96deb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:57.845282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241199c2-45ea-4b46-b16c-6246cdd96deb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:57.847467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:52:57.852768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327b09-720d-4271-9abe-931a6f63afc7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:57.853207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327b09-720d-4271-9abe-931a6f63afc7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:57.853322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf327b09-720d-4271-9abe-931a6f63afc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:57.921835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bf327b09-720d-4271-9abe-931a6f63afc7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:52:57.922958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bf327b09-720d-4271-9abe-931a6f63afc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:57.963170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:58.022661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:52:58.304429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241199c2-45ea-4b46-b16c-6246cdd96deb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:58.305963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-241199c2-45ea-4b46-b16c-6246cdd96deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:58.410956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241199c2-45ea-4b46-b16c-6246cdd96deb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:58.448964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:58.484905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:58.485506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:58.485783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:58.486052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:58.487723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:58.595095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:58.597305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d033ba5-75c1-44c7-aee7-123a8d3d4f1b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:58.615622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 734/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:00.874530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afb7fe-54f4-4027-b435-5bd3479ff77b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:00.875100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afb7fe-54f4-4027-b435-5bd3479ff77b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:00.875100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afb7fe-54f4-4027-b435-5bd3479ff77b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:00.876454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:00.881964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7bf1-e11c-438c-a4f4-0ae8b9b565f5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:00.882378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7bf1-e11c-438c-a4f4-0ae8b9b565f5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:00.882506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb7bf1-e11c-438c-a4f4-0ae8b9b565f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:00.948292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ddb7bf1-e11c-438c-a4f4-0ae8b9b565f5 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:00.949491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ddb7bf1-e11c-438c-a4f4-0ae8b9b565f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:00.967086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:01.046828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:01.337066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afb7fe-54f4-4027-b435-5bd3479ff77b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:01.338618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f8afb7fe-54f4-4027-b435-5bd3479ff77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:01.442167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afb7fe-54f4-4027-b435-5bd3479ff77b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:01.510119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:01.517296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:01.518503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:01.518780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:01.519050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:01.520707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:01.628109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:01.629493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabafec-1807-4057-b90a-223377dfad8a service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:01.648219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 732/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:03.834306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85837b3a-1da0-4c88-a80e-30c70effd543 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:03.834836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85837b3a-1da0-4c88-a80e-30c70effd543 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:03.834952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85837b3a-1da0-4c88-a80e-30c70effd543 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:03.836207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:03.841593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37b46-5c95-4bf6-92c2-6a7247837471 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:03.842025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37b46-5c95-4bf6-92c2-6a7247837471 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:03.842172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37b46-5c95-4bf6-92c2-6a7247837471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:03.908223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-80d37b46-5c95-4bf6-92c2-6a7247837471 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:03.909960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-80d37b46-5c95-4bf6-92c2-6a7247837471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:03.959603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:04.008322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:04.296634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85837b3a-1da0-4c88-a80e-30c70effd543 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:04.298252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85837b3a-1da0-4c88-a80e-30c70effd543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:04.401648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85837b3a-1da0-4c88-a80e-30c70effd543 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:04.449583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:04.476348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:04.476950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:04.477228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:04.477496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:04.479079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:04.587455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:04.588846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9caa5-a18e-4334-aa0a-c5672194d49a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:04.606377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 737/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:14.006782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=22 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:14.604162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=66 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:24.721825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd1a75-a6c0-4e7d-b061-ede63680fa3e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:24.722366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd1a75-a6c0-4e7d-b061-ede63680fa3e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:24.722366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd1a75-a6c0-4e7d-b061-ede63680fa3e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:24.723704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 734/1471] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:24.729116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c968-e95b-47d7-ae84-30ac1aac1a49 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:24.729542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c968-e95b-47d7-ae84-30ac1aac1a49 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:24.729690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3c968-e95b-47d7-ae84-30ac1aac1a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:24.797370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-02c3c968-e95b-47d7-ae84-30ac1aac1a49 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:24.798613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-02c3c968-e95b-47d7-ae84-30ac1aac1a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:24.894284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 738/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:25.197368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb664d-0440-4b18-8f89-2ad8553b039f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:25.197651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb664d-0440-4b18-8f89-2ad8553b039f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:25.197921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb664d-0440-4b18-8f89-2ad8553b039f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:25.199253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 735/1473] 10.222.0.57 () {54 vars in 962 bytes} [Thu Sep 15 12:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:25.204108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168b871-6ae8-435b-8bab-07a361e3cc84 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:25.204499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168b871-6ae8-435b-8bab-07a361e3cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:25.204776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168b871-6ae8-435b-8bab-07a361e3cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:25.281410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7168b871-6ae8-435b-8bab-07a361e3cc84 None None] MFA Rules not processed for user `6e29455534ee474899b4f470d2a55f94`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:25.283133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7168b871-6ae8-435b-8bab-07a361e3cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:25.304039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:25.359233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 739/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Sep 15 12:53:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:25.367035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb664d-0440-4b18-8f89-2ad8553b039f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:25.368591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2bfb664d-0440-4b18-8f89-2ad8553b039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:25.471322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb664d-0440-4b18-8f89-2ad8553b039f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:25.543734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:25.544350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:25.544628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:25.544894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:25.546462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:25.600209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:25.656111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:25.657540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81037-0aa9-4bbe-b754-dad87acb1f3c service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:25.676896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 736/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:53:27.908240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c01d3-eaa8-464f-b129-222df6fb94db None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:27.908835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c01d3-eaa8-464f-b129-222df6fb94db None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:27.908835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c01d3-eaa8-464f-b129-222df6fb94db None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:27.910170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 740/1476] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:27.915621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe8b4-6284-4d79-92fb-58253d43b4dc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:27.916054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe8b4-6284-4d79-92fb-58253d43b4dc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:27.916153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fe8b4-6284-4d79-92fb-58253d43b4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:27.983591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eb5fe8b4-6284-4d79-92fb-58253d43b4dc None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:27.984839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eb5fe8b4-6284-4d79-92fb-58253d43b4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:28.082540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 737/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:28.359358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c01d3-eaa8-464f-b129-222df6fb94db None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:28.361681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6a0c01d3-eaa8-464f-b129-222df6fb94db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:28.486740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c01d3-eaa8-464f-b129-222df6fb94db None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:28.536967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:28.581603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:28.582208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:28.582489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:28.582762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:28.583389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:28.585139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:53:28.585139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:53:28.585139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:53:28.586276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4d832-921f-472c-8e7f-ef14291ba7e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:28.604919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 741/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Thu Sep 15 12:53:28 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:53:30.880077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b906d-a019-4f87-9412-d3f86fe74890 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:30.880690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b906d-a019-4f87-9412-d3f86fe74890 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:30.881004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b906d-a019-4f87-9412-d3f86fe74890 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:30.883432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 738/1479] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:30.891145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c24a8-9435-4d07-ab48-7ee1d625cf6e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:30.892130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c24a8-9435-4d07-ab48-7ee1d625cf6e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:30.892252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c24a8-9435-4d07-ab48-7ee1d625cf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:30.959872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-761c24a8-9435-4d07-ab48-7ee1d625cf6e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:30.961118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-761c24a8-9435-4d07-ab48-7ee1d625cf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:31.057161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 742/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:31.356210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b906d-a019-4f87-9412-d3f86fe74890 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:31.357764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-326b906d-a019-4f87-9412-d3f86fe74890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:31.367862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:31.505929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b906d-a019-4f87-9412-d3f86fe74890 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:31.586485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:31.618413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:31.618953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:31.619236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:31.619505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:31.621121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:31.713584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:31.715190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb7c52-d634-419a-b5ff-c436f26ef607 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:31.733098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 739/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:34.075636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86067665-cd45-4224-a96f-6c2957569c50 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:34.076195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86067665-cd45-4224-a96f-6c2957569c50 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:34.076325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86067665-cd45-4224-a96f-6c2957569c50 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:34.077615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 743/1482] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:34.083499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c14f28-01e5-44d5-b0ac-9a537e53c97f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:34.083969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c14f28-01e5-44d5-b0ac-9a537e53c97f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:34.084180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c14f28-01e5-44d5-b0ac-9a537e53c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:34.149741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b4c14f28-01e5-44d5-b0ac-9a537e53c97f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:34.150959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b4c14f28-01e5-44d5-b0ac-9a537e53c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:34.205545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:34.257419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 740/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:34.547857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86067665-cd45-4224-a96f-6c2957569c50 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:34.549519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86067665-cd45-4224-a96f-6c2957569c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:34.573680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:34.655986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86067665-cd45-4224-a96f-6c2957569c50 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:34.731629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:34.732296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:34.732696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:34.732972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:34.734591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:34.808943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:34.843978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:34.846407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4833df-13da-413a-a62b-4b0c0fd94f87 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:34.864132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 744/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:37.074570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b50505-078f-423b-8e7a-b85925d354d0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:37.075191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b50505-078f-423b-8e7a-b85925d354d0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:37.075191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b50505-078f-423b-8e7a-b85925d354d0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:37.076497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 741/1485] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:37.083158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143905e-2478-4d6c-b30e-87a5cdad3564 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:37.083611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143905e-2478-4d6c-b30e-87a5cdad3564 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:37.083710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143905e-2478-4d6c-b30e-87a5cdad3564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:37.150218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8143905e-2478-4d6c-b30e-87a5cdad3564 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:37.151430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8143905e-2478-4d6c-b30e-87a5cdad3564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:37.247156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 745/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:37.537618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b50505-078f-423b-8e7a-b85925d354d0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:37.539175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-55b50505-078f-423b-8e7a-b85925d354d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:37.642612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b50505-078f-423b-8e7a-b85925d354d0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:37.714982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:37.716526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:37.717193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:37.717648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:37.718131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:37.720104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:37.832025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:37.833499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09b165-e517-4f7f-9cb4-f4eacedcc8e6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:37.851954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 742/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:40.148783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42628310-b07f-44ad-9dac-182782102dd4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:40.149301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42628310-b07f-44ad-9dac-182782102dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:40.149404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42628310-b07f-44ad-9dac-182782102dd4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:40.150680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 746/1488] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:40.155889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040d9e8-77bd-4e00-833f-116815fc74a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:40.156344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040d9e8-77bd-4e00-833f-116815fc74a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:40.156455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040d9e8-77bd-4e00-833f-116815fc74a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:40.223454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2040d9e8-77bd-4e00-833f-116815fc74a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:40.224685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2040d9e8-77bd-4e00-833f-116815fc74a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:40.278157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:40.324294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 743/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:40.620030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42628310-b07f-44ad-9dac-182782102dd4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:40.621626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-42628310-b07f-44ad-9dac-182782102dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:40.658591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:40.726184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42628310-b07f-44ad-9dac-182782102dd4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:40.800875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:40.801478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:40.801759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:40.802039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:40.803627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:40.884978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:40.915811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:40.917265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540147d-3be5-49d2-8a5e-a017f4e6ff2f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:40.935690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 747/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:43.384703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5901e8b-e641-4318-a538-b7e6361f71d5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:43.385254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5901e8b-e641-4318-a538-b7e6361f71d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:43.385254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5901e8b-e641-4318-a538-b7e6361f71d5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:43.386506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 744/1491] 10.222.0.57 () {54 vars in 927 bytes} [Thu Sep 15 12:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:53:43.393094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1ac3-174f-45b8-bb79-b3b1bb4fef44 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:43.393501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1ac3-174f-45b8-bb79-b3b1bb4fef44 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:43.393642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1ac3-174f-45b8-bb79-b3b1bb4fef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:43.458896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c2cb1ac3-174f-45b8-bb79-b3b1bb4fef44 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:53:43.460093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c2cb1ac3-174f-45b8-bb79-b3b1bb4fef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:43.556459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 748/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Thu Sep 15 12:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:53:43.852556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5901e8b-e641-4318-a538-b7e6361f71d5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:53:43.854109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d5901e8b-e641-4318-a538-b7e6361f71d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:43.956500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5901e8b-e641-4318-a538-b7e6361f71d5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:53:44.029370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:53:44.029977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:53:44.030258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:53:44.030530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:53:44.032815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:53:44.042204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:44.143697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:53:44.145235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa08bb-c98b-4699-ba70-b6bd84ca6885 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:53:44.162976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 745/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:53:53.554955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=82 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:53:54.160720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=50 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:03.261942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:03.262525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:03.262525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:03.338846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:55:03.340063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:03.436702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 749/1494] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:55:03.444688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:03.445168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:03.445276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:03.520817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:55:03.522033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:03.618092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 746/1495] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:55:03.622827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:03.624249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:03.635552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f40749-5080-43d5-96b7-184ee1563058 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:03.697812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:03.732787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:03.733404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:03.733683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:03.733959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:03.734590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:55:03.734888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:03.735898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:03.735898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:55:03.735898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:55:03.736812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:03.748340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 750/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:55:03.756565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:03.766011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b453ed-9bf6-4217-b10c-3af74d440f56 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:03.828466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:03.862513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:03.863111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:03.863391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:03.863686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:03.864341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:03.866109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:03.866109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:55:03.866109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:55:03.867336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:03.923549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 747/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:55:03 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:55:03.927803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:03.929379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:03.963579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f062-e644-4d9d-8ba2-33c48a5e8ff5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:04.056255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:04.056879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.057157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.057428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.058056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:04.059061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:04.059061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:55:04.059061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:55:04.060429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:04.103209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 751/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:55:03 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:55:04.107641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:04.109384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:04.172126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d2605-da38-44de-a990-6d80d1569a50 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:04.255417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:04.267822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:04.268464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.268744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.269038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.269703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:04.270813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:04.270813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:55:04.270813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:55:04.271702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:04.284129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 748/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:55:04.288211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:04.289790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:04.301349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25563643-a355-437e-b19b-ca3fe81d86aa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:04.327118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:04.397930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:04.398539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.398817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.399087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.399717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:04.400744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:04.400744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:55:04.400744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:55:04.401624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac64193-2e86-4910-93c5-ec1b8b626a7d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:04.414299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 752/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:55:04.418470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:04.421493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:04.454400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6d49d-f02c-472e-9ffd-24da4b770961 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:04.547935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:04.548534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.548811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.549077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] PATH_INFO: `/v3/projects/c652def352d44f178876c29dd3b2a86c/users/d0ae5ca4a7ea445793453373089d9bd1/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.571749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c652def352d44f178876c29dd3b2a86c, user_id=d0ae5ca4a7ea445793453373089d9bd1, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:04.572809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:04.572809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:55:04.572809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:55:04.574758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b3ab4-967c-426a-8969-8785c6e7f88e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:04.600583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 749/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:55:04 2022] PUT /identity/v3/projects/c652def352d44f178876c29dd3b2a86c/users/d0ae5ca4a7ea445793453373089d9bd1/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:55:04.612941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1556c1-4230-4f62-854c-a079b84fefc2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.613276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1556c1-4230-4f62-854c-a079b84fefc2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.613555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1556c1-4230-4f62-854c-a079b84fefc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.671596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1e1556c1-4230-4f62-854c-a079b84fefc2 None None] MFA Rules not processed for user `d0ae5ca4a7ea445793453373089d9bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:55:04.672850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1e1556c1-4230-4f62-854c-a079b84fefc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:04.749644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 753/1502] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:55:04.762845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755dc25-241c-4490-97f4-a476cdfdcd05 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.763136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755dc25-241c-4490-97f4-a476cdfdcd05 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.763425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755dc25-241c-4490-97f4-a476cdfdcd05 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.764796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 750/1503] 10.222.0.57 () {54 vars in 964 bytes} [Thu Sep 15 12:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:55:04.769823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce1b8-d77f-4e74-97dc-410309dcafc2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:04.770117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce1b8-d77f-4e74-97dc-410309dcafc2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:04.770396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce1b8-d77f-4e74-97dc-410309dcafc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:04.802291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:04.848414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-84bce1b8-d77f-4e74-97dc-410309dcafc2 None None] MFA Rules not processed for user `8d2458e0ed4b4fceb437ac5c7b43d598`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:55:04.849624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84bce1b8-d77f-4e74-97dc-410309dcafc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:04.926377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 754/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Thu Sep 15 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:55:04.934216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755dc25-241c-4490-97f4-a476cdfdcd05 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:04.936436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e755dc25-241c-4490-97f4-a476cdfdcd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:04.987648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:05.160542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755dc25-241c-4490-97f4-a476cdfdcd05 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:05.233612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:05.234218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:05.234501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:05.234776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:05.236406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:05.334961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:05.336426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:05.339843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:05.354619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 751/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:55:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:55:05.401083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea536-f4e6-45de-b3d3-d8e84283ea93 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:05.401541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea536-f4e6-45de-b3d3-d8e84283ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:05.401638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea536-f4e6-45de-b3d3-d8e84283ea93 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:05.402887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 755/1506] 10.222.0.57 () {52 vars in 830 bytes} [Thu Sep 15 12:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:55:05.406347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:05.407544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:05.503648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03addc21-fca3-401a-aacd-4f4b5968dc3d service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:05.596450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:05.597077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:05.597358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:05.597631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] PATH_INFO: `/v3/users/d0ae5ca4a7ea445793453373089d9bd1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:05.611302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ae5ca4a7ea445793453373089d9bd1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:05.612261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:05.612261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:55:05.612261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:55:05.613492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:05.623154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:05.679737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ae5ca4a7ea445793453373089d9bd1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:05.694296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ae5ca4a7ea445793453373089d9bd1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:05.717258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ae5ca4a7ea445793453373089d9bd1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:05.717861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ae5ca4a7ea445793453373089d9bd1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:05.726753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ae5ca4a7ea445793453373089d9bd1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:05.733664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 752/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:55:05 2022] DELETE /identity/v3/users/d0ae5ca4a7ea445793453373089d9bd1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:55:05.737927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:05.738233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:05.738508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:05.797525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:55:05.798748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:05.874867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:05.887405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 756/1508] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:55:05.897391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:05.898610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:06.002976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ea9f4-71e6-4786-8053-6bbd1e2c9818 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:06.058732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:06.079667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:06.080299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:06.080584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:06.080853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:06.082467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:06.196528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:06.197903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be95b77-0423-4160-8d8e-510f0f4add88 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:06.217083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 753/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:55:06.597243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:55:06.598483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:55:06.611983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627b14b-fd7c-4236-9ff4-415cd8713410 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:55:06.723885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:55:06.724480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:55:06.724762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:55:06.725056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] PATH_INFO: `/v3/projects/c652def352d44f178876c29dd3b2a86c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:55:06.737228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c652def352d44f178876c29dd3b2a86c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:55:06.738332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:55:06.738332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:55:06.738332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:55:06.739686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:55:06.798100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c652def352d44f178876c29dd3b2a86c', 'request_id': 'req-0b8b5241-429a-41e0-94d1-c30ff187005d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:06.803927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c652def352d44f178876c29dd3b2a86c', 'request_id': 'req-0b8b5241-429a-41e0-94d1-c30ff187005d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:06.818913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c652def352d44f178876c29dd3b2a86c', 'request_id': 'req-0b8b5241-429a-41e0-94d1-c30ff187005d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:06.845322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] The token cache is being invalidate because project c652def352d44f178876c29dd3b2a86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:55:06.845615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:06.846047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0b8b5241-429a-41e0-94d1-c30ff187005d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:55:06.847422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 757/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:55:06 2022] DELETE /identity/v3/projects/c652def352d44f178876c29dd3b2a86c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:55:16.214149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=62 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:55:16.843290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:56:29.097878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b121ded-1749-48f0-97dd-25c1db36455b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:29.098393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b121ded-1749-48f0-97dd-25c1db36455b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:29.098499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b121ded-1749-48f0-97dd-25c1db36455b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:29.099766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:29.104628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73199d-258f-4cf2-bbab-54ccc4ee92e8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:29.105212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73199d-258f-4cf2-bbab-54ccc4ee92e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:29.105502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73199d-258f-4cf2-bbab-54ccc4ee92e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:29.186076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8d73199d-258f-4cf2-bbab-54ccc4ee92e8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:29.187265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8d73199d-258f-4cf2-bbab-54ccc4ee92e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:29.290917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:29.446269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b121ded-1749-48f0-97dd-25c1db36455b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:29.448398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9b121ded-1749-48f0-97dd-25c1db36455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:29.566308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b121ded-1749-48f0-97dd-25c1db36455b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:29.571753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbb591-850c-4ad4-8ae8-aad92c0e5de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:29.572397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb591-850c-4ad4-8ae8-aad92c0e5de6 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:29.572681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb591-850c-4ad4-8ae8-aad92c0e5de6 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:29.572954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbb591-850c-4ad4-8ae8-aad92c0e5de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:29.581591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb591-850c-4ad4-8ae8-aad92c0e5de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:29.583164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbb591-850c-4ad4-8ae8-aad92c0e5de6 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:29.593226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 755/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:30.672655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44d95-3516-4c71-bdd5-db18d6403ffa None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:30.673225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44d95-3516-4c71-bdd5-db18d6403ffa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:30.673225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a44d95-3516-4c71-bdd5-db18d6403ffa None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:30.674574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:30.680759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73036c84-3d3c-4f4b-aebe-5ee868bc0f06 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:30.681179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73036c84-3d3c-4f4b-aebe-5ee868bc0f06 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:30.681289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73036c84-3d3c-4f4b-aebe-5ee868bc0f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:30.701976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-73036c84-3d3c-4f4b-aebe-5ee868bc0f06 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:30.703182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-73036c84-3d3c-4f4b-aebe-5ee868bc0f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:30.710986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:30.864490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44d95-3516-4c71-bdd5-db18d6403ffa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:30.879287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a44d95-3516-4c71-bdd5-db18d6403ffa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:30.884777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65e052-e141-42a4-9230-5a26df486263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:30.885376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65e052-e141-42a4-9230-5a26df486263 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:30.885658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65e052-e141-42a4-9230-5a26df486263 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:30.885929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65e052-e141-42a4-9230-5a26df486263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:30.894296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65e052-e141-42a4-9230-5a26df486263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:30.895873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65e052-e141-42a4-9230-5a26df486263 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:30.905342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 760/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:30.993625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9b410-986a-4cbd-8565-5b0ef29945d8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:30.994228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9b410-986a-4cbd-8565-5b0ef29945d8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:30.994509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9b410-986a-4cbd-8565-5b0ef29945d8 None None] PATH_INFO: `/v3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:30.996995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 757/1517] 10.222.0.27 () {54 vars in 925 bytes} [Thu Sep 15 12:56:30 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:56:31.004258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dc317-d113-4696-a4ae-20f0ea2886bc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:31.004585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dc317-d113-4696-a4ae-20f0ea2886bc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:31.004864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4dc317-d113-4696-a4ae-20f0ea2886bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:31.076612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fd4dc317-d113-4696-a4ae-20f0ea2886bc None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:31.077829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fd4dc317-d113-4696-a4ae-20f0ea2886bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:31.102412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:56:31.170506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 761/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Thu Sep 15 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:56:39.898466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848567f-b673-4937-bfa7-2c504e0750c4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:39.898950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848567f-b673-4937-bfa7-2c504e0750c4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:39.899068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848567f-b673-4937-bfa7-2c504e0750c4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:39.900318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 758/1519] 10.222.0.57 () {54 vars in 961 bytes} [Thu Sep 15 12:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:39.905564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354eaf3-edba-46a6-b8a0-5f64458b4101 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:39.906012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354eaf3-edba-46a6-b8a0-5f64458b4101 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:39.906112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354eaf3-edba-46a6-b8a0-5f64458b4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:39.925862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a354eaf3-edba-46a6-b8a0-5f64458b4101 None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:39.927060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a354eaf3-edba-46a6-b8a0-5f64458b4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:39.936279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 762/1520] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Sep 15 12:56:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:39.943974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848567f-b673-4937-bfa7-2c504e0750c4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:39.957632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848567f-b673-4937-bfa7-2c504e0750c4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:39.962552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374b47d-cfcf-4e34-9931-d4762f85eab8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:39.963143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b47d-cfcf-4e34-9931-d4762f85eab8 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:39.963423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b47d-cfcf-4e34-9931-d4762f85eab8 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:39.963695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b47d-cfcf-4e34-9931-d4762f85eab8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:39.972178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374b47d-cfcf-4e34-9931-d4762f85eab8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:39.973766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374b47d-cfcf-4e34-9931-d4762f85eab8 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:39.983958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 759/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:42.430629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064722-d631-4b11-9aaa-1decd59c7981 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:42.431196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064722-d631-4b11-9aaa-1decd59c7981 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:42.431196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064722-d631-4b11-9aaa-1decd59c7981 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:42.432472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:42.439252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff570127-b9e4-4be4-a44d-b5d745e4f10d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:42.439692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff570127-b9e4-4be4-a44d-b5d745e4f10d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:42.439807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff570127-b9e4-4be4-a44d-b5d745e4f10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:42.458928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ff570127-b9e4-4be4-a44d-b5d745e4f10d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:42.460126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ff570127-b9e4-4be4-a44d-b5d745e4f10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:42.468220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:42.627953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d064722-d631-4b11-9aaa-1decd59c7981 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:42.643712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d064722-d631-4b11-9aaa-1decd59c7981 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:42.649119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6aa7f-5f4e-45e6-a069-d365c1024c7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:42.649712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6aa7f-5f4e-45e6-a069-d365c1024c7d service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:42.649990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6aa7f-5f4e-45e6-a069-d365c1024c7d service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:42.650259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6aa7f-5f4e-45e6-a069-d365c1024c7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:42.658649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6aa7f-5f4e-45e6-a069-d365c1024c7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:42.660259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6aa7f-5f4e-45e6-a069-d365c1024c7d service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:42.669770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 764/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:43.747911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d949d-aade-4f18-8d2e-29a87aac5056 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:43.748521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d949d-aade-4f18-8d2e-29a87aac5056 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:43.748521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d949d-aade-4f18-8d2e-29a87aac5056 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:43.749828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:43.756151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dded85-3011-45c7-9a4d-b7c318772e4e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:43.756649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dded85-3011-45c7-9a4d-b7c318772e4e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:43.756752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dded85-3011-45c7-9a4d-b7c318772e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:43.777444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-36dded85-3011-45c7-9a4d-b7c318772e4e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:43.778663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-36dded85-3011-45c7-9a4d-b7c318772e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:43.786587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:43.938941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d949d-aade-4f18-8d2e-29a87aac5056 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:43.953928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d949d-aade-4f18-8d2e-29a87aac5056 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:43.959401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719cec4-bf85-46b9-87bb-25e51c9211e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:43.959996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719cec4-bf85-46b9-87bb-25e51c9211e6 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:43.960292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719cec4-bf85-46b9-87bb-25e51c9211e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:43.960575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719cec4-bf85-46b9-87bb-25e51c9211e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:43.969593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719cec4-bf85-46b9-87bb-25e51c9211e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:43.971166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719cec4-bf85-46b9-87bb-25e51c9211e6 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:43.981114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 762/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:47.819796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23c28f-71e8-4017-955c-ee851e5b222e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:47.820322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23c28f-71e8-4017-955c-ee851e5b222e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:47.820435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23c28f-71e8-4017-955c-ee851e5b222e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:47.821719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:47.827670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2de9b-68a6-4451-a49f-f1da633a56fa None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:47.828088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2de9b-68a6-4451-a49f-f1da633a56fa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:47.828203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2de9b-68a6-4451-a49f-f1da633a56fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:47.847390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-18b2de9b-68a6-4451-a49f-f1da633a56fa None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:47.848608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-18b2de9b-68a6-4451-a49f-f1da633a56fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:47.856806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:48.144263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23c28f-71e8-4017-955c-ee851e5b222e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:48.145985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a23c28f-71e8-4017-955c-ee851e5b222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:48.225702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23c28f-71e8-4017-955c-ee851e5b222e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:48.229316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b0f60-dd89-45d7-bb4a-7c628add7700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:48.229921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b0f60-dd89-45d7-bb4a-7c628add7700 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:48.230204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b0f60-dd89-45d7-bb4a-7c628add7700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:48.230477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b0f60-dd89-45d7-bb4a-7c628add7700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:48.239258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b0f60-dd89-45d7-bb4a-7c628add7700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:48.240716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b0f60-dd89-45d7-bb4a-7c628add7700 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:48.250085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 767/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:50.394644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84d605-3428-49e1-9cc6-1f312f54f606 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:50.395160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84d605-3428-49e1-9cc6-1f312f54f606 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:50.395160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84d605-3428-49e1-9cc6-1f312f54f606 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:50.396533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:50.401911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfe418-f24b-4eb4-9bab-52018e55da0b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:50.402335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfe418-f24b-4eb4-9bab-52018e55da0b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:50.402476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfe418-f24b-4eb4-9bab-52018e55da0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:50.422283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-50bfe418-f24b-4eb4-9bab-52018e55da0b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:50.423481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50bfe418-f24b-4eb4-9bab-52018e55da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:50.431678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:56:50.719166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84d605-3428-49e1-9cc6-1f312f54f606 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:50.730333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84d605-3428-49e1-9cc6-1f312f54f606 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:50.733834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680c41f-cb9b-4e14-a00b-240bc3f62915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:50.734425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c41f-cb9b-4e14-a00b-240bc3f62915 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:50.734705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c41f-cb9b-4e14-a00b-240bc3f62915 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:50.734980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680c41f-cb9b-4e14-a00b-240bc3f62915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:50.743725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c41f-cb9b-4e14-a00b-240bc3f62915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:50.745835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680c41f-cb9b-4e14-a00b-240bc3f62915 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:50.755083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 765/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:54.336850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17001c55-ee0f-4162-a607-fadffc4ba3fd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:54.337423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17001c55-ee0f-4162-a607-fadffc4ba3fd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:54.337423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17001c55-ee0f-4162-a607-fadffc4ba3fd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:54.339700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:54.345076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10c28-cb6a-4ef7-9035-65bd6abac2db None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:54.345527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10c28-cb6a-4ef7-9035-65bd6abac2db None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:54.345631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10c28-cb6a-4ef7-9035-65bd6abac2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:54.366045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b1d10c28-cb6a-4ef7-9035-65bd6abac2db None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:54.367160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b1d10c28-cb6a-4ef7-9035-65bd6abac2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:54.375261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:56:54.679172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17001c55-ee0f-4162-a607-fadffc4ba3fd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:54.680843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17001c55-ee0f-4162-a607-fadffc4ba3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:54.749959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:56:54.764535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17001c55-ee0f-4162-a607-fadffc4ba3fd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:54.768401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232ae26-d4fa-4461-bc63-ecd3e33da1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:54.769018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232ae26-d4fa-4461-bc63-ecd3e33da1df service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:54.769301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232ae26-d4fa-4461-bc63-ecd3e33da1df service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:54.769577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232ae26-d4fa-4461-bc63-ecd3e33da1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:54.778643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232ae26-d4fa-4461-bc63-ecd3e33da1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:54.780071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232ae26-d4fa-4461-bc63-ecd3e33da1df service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:54.789555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 770/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:56:56.942100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f58df-61e4-46ac-b263-52a06a46ce5b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:56.942642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f58df-61e4-46ac-b263-52a06a46ce5b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:56.942642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f58df-61e4-46ac-b263-52a06a46ce5b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:56.943973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:56:56.949770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808f8c-24f8-4b8f-8ed2-551686314383 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:56.950198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808f8c-24f8-4b8f-8ed2-551686314383 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:56.950353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808f8c-24f8-4b8f-8ed2-551686314383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:56.970661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d2808f8c-24f8-4b8f-8ed2-551686314383 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:56:56.971888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2808f8c-24f8-4b8f-8ed2-551686314383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:56.980640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:56:57.284940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f58df-61e4-46ac-b263-52a06a46ce5b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:56:57.286674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d90f58df-61e4-46ac-b263-52a06a46ce5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:56:57.294732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:56:57.301042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f58df-61e4-46ac-b263-52a06a46ce5b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:56:57.304562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047d5f0-b8ae-447d-aee7-75bed4f40089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:56:57.305159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d5f0-b8ae-447d-aee7-75bed4f40089 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:56:57.305439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d5f0-b8ae-447d-aee7-75bed4f40089 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:56:57.305718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d5f0-b8ae-447d-aee7-75bed4f40089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:56:57.315268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047d5f0-b8ae-447d-aee7-75bed4f40089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:56:57.316728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047d5f0-b8ae-447d-aee7-75bed4f40089 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:56:57.326232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 768/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:57:06.965759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=11 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:57:07.321868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:21.928623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f94a-7524-48fb-b436-9d7e323fb7e9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:21.929747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f94a-7524-48fb-b436-9d7e323fb7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:21.930224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f94a-7524-48fb-b436-9d7e323fb7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:21.956957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6378f94a-7524-48fb-b436-9d7e323fb7e9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:21.958568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6378f94a-7524-48fb-b436-9d7e323fb7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:21.971912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 772/1540] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:21.982198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:21.982795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:21.982795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.020873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.022347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.040130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d44b50-5ba7-4d24-858f-7ed87a2a6990 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.040946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d44b50-5ba7-4d24-858f-7ed87a2a6990 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.042010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d44b50-5ba7-4d24-858f-7ed87a2a6990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.043076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 769/1541] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.047745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.070727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e3289-c948-4a73-96aa-dbb6ca3173fd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.078603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-75d44b50-5ba7-4d24-858f-7ed87a2a6990 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.079747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-75d44b50-5ba7-4d24-858f-7ed87a2a6990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.085278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.085912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.086192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.086465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.087122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.087428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.090974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:58:22.090974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:58:22.090974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:58:22.091277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ca3eb-657b-45ef-849e-08109ec09d49 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.112949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 773/1542] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:22.121837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.122148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.122400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.125284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 770/1543] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:58:22.127503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747a193-333e-4624-b27b-8b0ed95ce7d7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.127793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747a193-333e-4624-b27b-8b0ed95ce7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.128070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747a193-333e-4624-b27b-8b0ed95ce7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.165401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.167482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.177605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0747a193-333e-4624-b27b-8b0ed95ce7d7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.178828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0747a193-333e-4624-b27b-8b0ed95ce7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.196191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 774/1544] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.197101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.199553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 771/1545] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:22.204539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33220-98cf-45ad-a0e3-07282e250fd3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.204970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33220-98cf-45ad-a0e3-07282e250fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.205252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33220-98cf-45ad-a0e3-07282e250fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.208034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7454bfb-81e0-4684-9624-78a4e762d369 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.220202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.220848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.221129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.221405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.222043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.223065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:58:22.223065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:58:22.223065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:58:22.224293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.238680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3ca33220-98cf-45ad-a0e3-07282e250fd3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.240141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ca33220-98cf-45ad-a0e3-07282e250fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.259018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 772/1546] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.260480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe2a68-90e4-4547-95c0-4367fc322147 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.260805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe2a68-90e4-4547-95c0-4367fc322147 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.261078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe2a68-90e4-4547-95c0-4367fc322147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.297699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8dfe2a68-90e4-4547-95c0-4367fc322147 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.302044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 775/1547] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.302186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8dfe2a68-90e4-4547-95c0-4367fc322147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.302281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.364977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 773/1548] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.366156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22ff4-ac86-46d6-91df-b0c92564b3e8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.366443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22ff4-ac86-46d6-91df-b0c92564b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.366721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22ff4-ac86-46d6-91df-b0c92564b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.371878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c050df-a38b-4862-8de6-fceae7e744e8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.376967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.377582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.377868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.378145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.378788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.379095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.380740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172120b-1d03-40dd-8616-abfcc5e7c149 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.386103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b1f22ff4-ac86-46d6-91df-b0c92564b3e8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.387302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b1f22ff4-ac86-46d6-91df-b0c92564b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.392953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 776/1549] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.394179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae15f5-8f77-4819-9eb1-5a0def5d7ca4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.394428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae15f5-8f77-4819-9eb1-5a0def5d7ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.394709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae15f5-8f77-4819-9eb1-5a0def5d7ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.399015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 774/1550] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.399015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.399015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.399015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.417247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-03ae15f5-8f77-4819-9eb1-5a0def5d7ca4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.417533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.418460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-03ae15f5-8f77-4819-9eb1-5a0def5d7ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.418592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.426438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 777/1551] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.426662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 775/1552] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.427397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.427624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e4bc1-dbf6-4f2b-a163-884b6018fc0f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.427912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e4bc1-dbf6-4f2b-a163-884b6018fc0f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.428186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e4bc1-dbf6-4f2b-a163-884b6018fc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.438229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d5f7-0124-4555-9adf-d8d7ea2183b8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.442970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.444007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.444316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.444690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.445326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.446369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:58:22.446369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:58:22.446369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:58:22.447579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c846ac-9386-41b6-a3c1-8d491d662113 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.449876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-393e4bc1-dbf6-4f2b-a163-884b6018fc0f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.451061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-393e4bc1-dbf6-4f2b-a163-884b6018fc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.459163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 778/1553] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.460563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.460879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.461102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.476918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 776/1554] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.478200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.478491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.478765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.481425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.482268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.490233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 779/1555] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.491019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.500154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.501611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23d88-275e-4ae5-8840-7243240c50db None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.501741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.506241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.506778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.507063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.507338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.507972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.508298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.509921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2024543a-845f-47e2-9b1a-ff1eb5469d19 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.514828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 777/1556] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.514828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.523120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd4a04-2544-4c90-be87-1d6abdee1843 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.532709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 780/1557] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.532709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.532709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.532709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.533156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ef4ed-37d6-477a-9245-b7fbf33d1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.533156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ef4ed-37d6-477a-9245-b7fbf33d1d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.533156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ef4ed-37d6-477a-9245-b7fbf33d1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.533156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ef4ed-37d6-477a-9245-b7fbf33d1d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.533504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ef4ed-37d6-477a-9245-b7fbf33d1d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.533504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ef4ed-37d6-477a-9245-b7fbf33d1d11 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.551948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.553303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.562706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 781/1558] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.563515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.568214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 778/1559] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.568214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.568214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.568214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.573996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00db8db-4dc5-4330-87b2-683a4fc90917 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.587032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.588226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.633418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.633972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.634256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.634531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.635196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.635527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.637523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290cf4a-87f2-4990-a169-d51e08635380 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.644451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:22.649022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 779/1560] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.649871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.650496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 782/1561] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.652398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.653377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.653694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.666335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe386fe-8219-484a-b451-58bce76d8073 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.671682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.672330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.672702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.672980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.673621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.674012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:22.675220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:22.675340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.677370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:22.684462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 783/1562] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:22.685341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.690151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 780/1563] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:22.690151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.695051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed303d-6f08-4ddc-ab2d-2b361ae2f760 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.698438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0278417-1d25-45e3-a1d8-52e8b9968fa6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.699839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.700475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.700762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.701038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.701701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.702007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.703336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.703597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.703922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.704204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.704514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.705148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.707035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.715442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 784/1564] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.716247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.726108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5fce14-bd62-432f-b6d8-af38863649f4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.731995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.732737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.733024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.733300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.734079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.734391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.736009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.740869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 781/1565] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.741802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.749086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 785/1566] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.750044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.759851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9580225-8593-48e9-9a19-80ea85dfceac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.808482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81c4b9-6a21-45b0-ab86-d4ba3023da9d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.813401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.814013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.814293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.814563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.815230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.817528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.821366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.821985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.822270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.822544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.823911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.824211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.825830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.845376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 786/1567] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.845376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.845682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 782/1568] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.845682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.850010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a2c30-4c7b-49bc-b3eb-e9e56e51f618 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.855470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.855906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0bdaa-1335-42ad-844f-780867cd5e47 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.856066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.856393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.856669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.857320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.859264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.860681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.861270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.861610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.861831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.862467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:22.862767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.864367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.877518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 783/1569] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:22.880965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.890171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 787/1570] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.890171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.896261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec500170-9533-40ad-8c70-9ad2abd18473 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.956010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d2a44-0d26-4ded-87e4-8698f20d0460 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.956662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.956662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.956932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.957210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.957841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.959441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.961092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.961700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.961981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.962253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.962876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.964985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.969612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 784/1571] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:22.970494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:22.980444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488efe5-530d-4f12-98fc-8cc3a8e8bd5b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:22.986078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:22.986724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:22.986938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:22.987207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:22.987826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:22.989788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:22.996581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 788/1572] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:22.996581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.012344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 785/1573] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.013240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.021332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:23.023114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2f09-fb97-491f-9a63-8e45a07ceafb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.051926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:23.063582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b786b3-0a00-465d-8bd2-6ad6e825df27 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.068647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.069273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.069556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.069830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.070460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.072398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.083168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.083784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.084064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.084371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.085048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.086681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.102477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 786/1574] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.102477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.108354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 789/1575] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:22 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.109234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.110188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b4e9b-179b-4fc6-86e1-a61783c13301 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.168420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.168944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.169222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.169522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.170148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.172046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.173019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2a81-09ae-430a-b862-5db22d0c5192 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.177647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.178256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.178538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.178810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.179432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.181477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.200060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 787/1576] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.201123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.209913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 790/1577] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.210820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.267208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844db14-3959-4916-b543-08bb77855104 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.271820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.272450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e555a4-21b8-4d6b-8f82-e784707c5ded admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.272622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.272739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.272984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.273603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.275519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.277204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.277852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.278138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.278410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] PATH_INFO: `/v3/projects/2e4c4dc894bd429b8f46ae444eb5ff7d/users/cd22d2df9ced4bb0961596385854422d/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.293609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4c4dc894bd429b8f46ae444eb5ff7d, user_id=cd22d2df9ced4bb0961596385854422d, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.294643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:58:23.294643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:58:23.294643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:58:23.296620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.297033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 788/1578] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.297821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.307608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ee41e-4bef-43b4-bfba-bf8aea199901 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.313174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 791/1579] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:23 2022] PUT /identity/v3/projects/2e4c4dc894bd429b8f46ae444eb5ff7d/users/cd22d2df9ced4bb0961596385854422d/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:23.317129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.334275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:23.347337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:23.366155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.366768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.367042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.367312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.367931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.369942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.382306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926f00c-812b-4d9b-bfa3-1b018eeb2fc0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.388185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.388839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.389121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.389440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.390076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.391976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.392207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 789/1580] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.393077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.403269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ca7e9-9f67-497b-83b6-73bad049fe5c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.406196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 792/1581] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.406901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.411458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.412061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.412362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.412664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.413307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.415643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.418622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a1858-7985-4411-b18c-e1b36fbf8207 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.423257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.423955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.424332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.424611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.425227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.426459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 790/1582] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.427125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.427231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.436900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1037ef5f-780f-410b-924f-a28ed6f13aff admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.442685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.443380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.443696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.443975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.444778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.446885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.455383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 793/1583] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.455383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.464602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a422b-bfc2-4de4-864b-9ac544bcc4ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.469429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bebf0c-b386-4a21-8640-21c4b8eaf6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.470026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bebf0c-b386-4a21-8640-21c4b8eaf6df admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.470322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bebf0c-b386-4a21-8640-21c4b8eaf6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.470465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 791/1584] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.470561 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bebf0c-b386-4a21-8640-21c4b8eaf6df admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.471177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.471278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bebf0c-b386-4a21-8640-21c4b8eaf6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.473249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bebf0c-b386-4a21-8640-21c4b8eaf6df admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.484026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0af271-308d-491d-97dc-e9e5ddd341eb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.488853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.489444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.489727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.490035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.490659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.492247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.497215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 794/1585] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:23.498283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.498569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.498955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.505311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 792/1586] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.505311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.513440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356a45d-1fdd-4a89-8561-ded13d397231 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.518373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.518955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.519234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.519531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.520150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.521701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:23.521838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.522909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:23.531184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 795/1587] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:23.531965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.533567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 793/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.534539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.542103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448aae9-b0ff-41b8-8cab-f64b8e75d8cf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.546491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e76c3-9125-4a0c-9013-0ccde7c36dea admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.546767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.547377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.547661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.547932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.548590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.550225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.551166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.551803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.552080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.552623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] PATH_INFO: `/v3/projects/de365ed54c494a3da402b036ca326abc/users/27f02bbd8abd45aaa3d5fd6070b5adc1/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.560613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 796/1589] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.561445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.567597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de365ed54c494a3da402b036ca326abc, user_id=27f02bbd8abd45aaa3d5fd6070b5adc1, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.570399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.571180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebccebc-d66d-4f78-8139-07a16395fa5e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.576829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.577524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.577872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.578260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.579002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.580717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.584940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 794/1590] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:23 2022] PUT /identity/v3/projects/de365ed54c494a3da402b036ca326abc/users/27f02bbd8abd45aaa3d5fd6070b5adc1/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:23.595862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.595990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 797/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.595990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.603841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0464fd9-25d8-4da3-a355-0504a805ef7d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.652580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99ddc-dfd8-4027-b554-7852c2a17e9a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.657289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.657898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.658181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.658458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.659096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.660744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.661487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.662103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.662382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.662661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.663336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.665054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.671275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 795/1592] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.672021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.676142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 798/1593] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.676865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.684559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72605de9-9f38-4fa0-936c-ec9b6bee4fd8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.687312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307606b1-476f-49d8-b0f7-7eeb46e52e97 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.689217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.689810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.690091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.690370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.690996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.692053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.692612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.692732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.692960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.693234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.693974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.695606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.702686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 796/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.703487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.709880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 799/1595] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.709880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.719546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf4255-b617-498b-9a8f-566fbfb4a25f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.724102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.724768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.725047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.725337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] PATH_INFO: `/v3/projects/6180da28285943f7a7b0900185df6127/users/2f0feb7134ee408b989b398a315850fe/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.740992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6180da28285943f7a7b0900185df6127, user_id=2f0feb7134ee408b989b398a315850fe, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.743777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.748199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:23.758662 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 800/1596] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:23 2022] PUT /identity/v3/projects/6180da28285943f7a7b0900185df6127/users/2f0feb7134ee408b989b398a315850fe/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:23.762782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.762905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf7294-2688-4239-829d-ccc991c75c58 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.771504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:23.811044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.811657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.811940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.812318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.827283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fc2b5-7109-4743-8382-b0f0c22ad40d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.832430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae7b2-eceb-4434-8745-5d3cc60a6aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.833037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae7b2-eceb-4434-8745-5d3cc60a6aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.833319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae7b2-eceb-4434-8745-5d3cc60a6aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.833591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae7b2-eceb-4434-8745-5d3cc60a6aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.834227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae7b2-eceb-4434-8745-5d3cc60a6aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.835815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ae7b2-eceb-4434-8745-5d3cc60a6aa6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.846702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 801/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.847933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.848314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.848599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.874021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.875608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.877864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] MFA Rules not processed for user `27f02bbd8abd45aaa3d5fd6070b5adc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:23.879073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:23.885550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 797/1598] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:23.886246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.895532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278c9fd-ce7f-47db-b923-15d31b88329b service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.900184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.901349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.901641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.901918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.902659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.904258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.914606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 798/1599] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.915238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.925949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3fa5f-a117-43e8-8a3d-371e26197364 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.931180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.931784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.932112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.932441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.933065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.934704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.949925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 799/1600] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:23.949925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.956610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 802/1601] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:23.957427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:23.963891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395e30-8734-4f1b-9f9f-af6efa9347ae admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.969178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b435f-6e4f-4c96-abe1-bf2e12f3d812 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:23.969954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaae93-c23c-4543-b1d3-04065db46675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.970963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaae93-c23c-4543-b1d3-04065db46675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.971350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaae93-c23c-4543-b1d3-04065db46675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.971640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaae93-c23c-4543-b1d3-04065db46675 admin admin] PATH_INFO: `/v3/projects/27776e610cf84b49af0fec4b95aa31f3/users/cff1145566f1408a9a22da9088581024/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.974074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:23.974740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:23.975022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:23.975415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] PATH_INFO: `/v3/projects/1741b69bb2be4ef08c76a4e0fb6e1487/users/204936c53844491682cef72ce215e035/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:23.989851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaae93-c23c-4543-b1d3-04065db46675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27776e610cf84b49af0fec4b95aa31f3, user_id=cff1145566f1408a9a22da9088581024, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.992745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaae93-c23c-4543-b1d3-04065db46675 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:23.993187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1741b69bb2be4ef08c76a4e0fb6e1487, user_id=204936c53844491682cef72ce215e035, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:23.996930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.007385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 800/1602] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:23 2022] PUT /identity/v3/projects/27776e610cf84b49af0fec4b95aa31f3/users/cff1145566f1408a9a22da9088581024/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:24.008685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.008971 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.009245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.015025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 803/1603] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:23 2022] PUT /identity/v3/projects/1741b69bb2be4ef08c76a4e0fb6e1487/users/204936c53844491682cef72ce215e035/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:24.016459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.030742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:24.031967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.107930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 801/1604] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:24.108901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.112433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c36dd3-0e30-4edd-a0b9-f4596f4f513e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.117807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:24.119174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.120817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.121263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.121521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] PATH_INFO: `/v3/projects/dc2cd7163bd9476891dfe836246ead22/users/082e5aa56ac147e6a8f8c797b269ff23/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.124709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f637142-54c5-4526-ab55-9bbf9578cc17 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.131305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.132378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.132733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.133106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] PATH_INFO: `/v3/projects/ffff4b9feef3437cb5529aeaaacc3f86/users/e2ef1277bf944722aa33077985fb4734/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.140831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2cd7163bd9476891dfe836246ead22, user_id=082e5aa56ac147e6a8f8c797b269ff23, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.145266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.148669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:24.156666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffff4b9feef3437cb5529aeaaacc3f86, user_id=e2ef1277bf944722aa33077985fb4734, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.159398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.166165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 804/1605] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:24 2022] PUT /identity/v3/projects/dc2cd7163bd9476891dfe836246ead22/users/082e5aa56ac147e6a8f8c797b269ff23/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:24.167900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.176603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 802/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:24 2022] PUT /identity/v3/projects/ffff4b9feef3437cb5529aeaaacc3f86/users/e2ef1277bf944722aa33077985fb4734/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:24.177697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.243019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a614538-e93d-4922-a6a1-f6e6066810c1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.253758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044f67f-f307-46d3-b736-d5de6b0a90b7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.264212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37aa19-6879-4a43-964e-bcfd9dc224e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.265327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37aa19-6879-4a43-964e-bcfd9dc224e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.265477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37aa19-6879-4a43-964e-bcfd9dc224e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.265916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37aa19-6879-4a43-964e-bcfd9dc224e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.267199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37aa19-6879-4a43-964e-bcfd9dc224e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.270416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37aa19-6879-4a43-964e-bcfd9dc224e4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.316436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 803/1607] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:24.318596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.318884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.319160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.329181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca7bc8-452b-4ff6-8513-98b868ca877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.331911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca7bc8-452b-4ff6-8513-98b868ca877e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.332479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca7bc8-452b-4ff6-8513-98b868ca877e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.333005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca7bc8-452b-4ff6-8513-98b868ca877e admin admin] PATH_INFO: `/v3/projects/29549a86d26043369f989b70d85596d2/users/21f78ba4b48249c688c3e3ed968d09d8/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.344211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:24.345986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.361892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca7bc8-452b-4ff6-8513-98b868ca877e admin admin] RBAC: Authorizing `identity:create_grant(project_id=29549a86d26043369f989b70d85596d2, user_id=21f78ba4b48249c688c3e3ed968d09d8, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.366083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca7bc8-452b-4ff6-8513-98b868ca877e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.381923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 805/1608] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:24 2022] PUT /identity/v3/projects/29549a86d26043369f989b70d85596d2/users/21f78ba4b48249c688c3e3ed968d09d8/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:24.383401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098a33-9bb9-43e7-a0ff-cac86806fc41 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.383757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098a33-9bb9-43e7-a0ff-cac86806fc41 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.384096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098a33-9bb9-43e7-a0ff-cac86806fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.420807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d1098a33-9bb9-43e7-a0ff-cac86806fc41 None None] MFA Rules not processed for user `204936c53844491682cef72ce215e035`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:24.422022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d1098a33-9bb9-43e7-a0ff-cac86806fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.426159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 804/1609] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:24.427216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.428705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.494264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fd5da-9041-402a-b3d2-e4f7a523830e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.507554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 806/1610] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:24.508904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0cf52-5e5f-4fc1-bf12-b7807c225157 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.509201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0cf52-5e5f-4fc1-bf12-b7807c225157 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.509486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0cf52-5e5f-4fc1-bf12-b7807c225157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.517047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:24.536229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bcc0cf52-5e5f-4fc1-bf12-b7807c225157 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:24.537611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bcc0cf52-5e5f-4fc1-bf12-b7807c225157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.549666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:24.571815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995149-5628-41f9-9d46-b985f3192535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.572617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995149-5628-41f9-9d46-b985f3192535 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.573167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995149-5628-41f9-9d46-b985f3192535 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.573314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995149-5628-41f9-9d46-b985f3192535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.635125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 807/1611] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:24.636847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.637292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.637552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.665123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995149-5628-41f9-9d46-b985f3192535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.666156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:24.667419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995149-5628-41f9-9d46-b985f3192535 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.668348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.682591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 808/1612] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:24.683356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.695988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 805/1613] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:24.696763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298de982-04e7-4aed-9d77-8d178f54c3de None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.700765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.700765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.701098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.703455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.704141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.704465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.704739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.705526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.707557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.749589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 809/1614] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/users => generated 289 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:24.750650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.756773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] MFA Rules not processed for user `21f78ba4b48249c688c3e3ed968d09d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:24.758299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:24.773364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a9585-ee82-4b54-8f0c-afa4bca247df admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.788233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c53b5a-44f1-49f1-9dc2-a70a2747b285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.788891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53b5a-44f1-49f1-9dc2-a70a2747b285 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.789180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53b5a-44f1-49f1-9dc2-a70a2747b285 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.789456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c53b5a-44f1-49f1-9dc2-a70a2747b285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.807909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c53b5a-44f1-49f1-9dc2-a70a2747b285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.809386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c53b5a-44f1-49f1-9dc2-a70a2747b285 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.831879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 810/1615] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:24.833222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79ad1-61bd-4bc0-9f56-2104dd032519 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.833513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79ad1-61bd-4bc0-9f56-2104dd032519 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.833792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79ad1-61bd-4bc0-9f56-2104dd032519 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.837369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 811/1616] 10.222.0.57 () {54 vars in 962 bytes} [Thu Sep 15 12:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:58:24.838200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c79ad1-61bd-4bc0-9f56-2104dd032519 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.859396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 806/1617] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:24.860647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.874311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e499d1b-121a-43d6-b8e0-2342992a76c2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.876729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c79ad1-61bd-4bc0-9f56-2104dd032519 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.880640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.881192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.881481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.881759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.889751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58f78-b560-4237-b91c-d380fe03ead8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.890389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58f78-b560-4237-b91c-d380fe03ead8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.890713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58f78-b560-4237-b91c-d380fe03ead8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.890948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58f78-b560-4237-b91c-d380fe03ead8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.898075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.899982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.922357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 807/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:24.923490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:24.929807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58f78-b560-4237-b91c-d380fe03ead8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.931776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58f78-b560-4237-b91c-d380fe03ead8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.937677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d15a-ea1b-48f8-9961-50cee790cd09 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:24.944227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a1a7e-1db5-4cb7-a9dc-816a86758596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:24.945269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a1a7e-1db5-4cb7-a9dc-816a86758596 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.945558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a1a7e-1db5-4cb7-a9dc-816a86758596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.945883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a1a7e-1db5-4cb7-a9dc-816a86758596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.947320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a1a7e-1db5-4cb7-a9dc-816a86758596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:24.950827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a1a7e-1db5-4cb7-a9dc-816a86758596 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:24.969085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 808/1619] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:24.970842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.971128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.971577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.978255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 812/1620] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:24.979671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a22b9a-ba6e-4a2f-8923-bdb3f7eb7d84 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:24.979800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a22b9a-ba6e-4a2f-8923-bdb3f7eb7d84 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:24.986253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a22b9a-ba6e-4a2f-8923-bdb3f7eb7d84 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:24.987969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 813/1621] 10.222.0.57 () {54 vars in 962 bytes} [Thu Sep 15 12:58:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:58:24.988633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a22b9a-ba6e-4a2f-8923-bdb3f7eb7d84 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:25.017171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a22b9a-ba6e-4a2f-8923-bdb3f7eb7d84 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:25.024696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47ab00-82aa-4e52-a6cf-7768a05c32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:25.025683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47ab00-82aa-4e52-a6cf-7768a05c32e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.026601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47ab00-82aa-4e52-a6cf-7768a05c32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.027234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47ab00-82aa-4e52-a6cf-7768a05c32e5 admin admin] PATH_INFO: `/v3/projects/a71da868d1874796b1910ec10bf8cd97/users/a5d45e5b20284dc292acc6259e596f45/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.047114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47ab00-82aa-4e52-a6cf-7768a05c32e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71da868d1874796b1910ec10bf8cd97, user_id=a5d45e5b20284dc292acc6259e596f45, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:25.051206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47ab00-82aa-4e52-a6cf-7768a05c32e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:25.074192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 814/1622] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:24 2022] PUT /identity/v3/projects/a71da868d1874796b1910ec10bf8cd97/users/a5d45e5b20284dc292acc6259e596f45/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:25.076843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.077138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.077462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.089420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:25.092291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:25.129470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:25.130872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:25.140109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:25.258883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 815/1623] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Sep 15 12:58:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:25.261238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:25.290983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 809/1624] 10.222.0.57 () {58 vars in 1059 bytes} [Thu Sep 15 12:58:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:25.295084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:25.311964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:25.326748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f0247-7466-4ad3-8073-6376fca65397 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:25.340298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:25.341089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.341591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.341957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.399399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7a73e-9ac6-4017-a404-c999d005151e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:25.411768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:25.414934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.417646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.418061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.419256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:25.423395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:25.461431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 816/1625] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:25.462603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:25.473656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:25.476613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:25.490371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c4d5f-7884-497c-b281-c151def26202 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:25.501501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:25.502843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.503627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.504446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.510033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 810/1626] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:25.529790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:25.564939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21dfaf-ef2d-43d6-b34a-b09c2bd3b16d service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:25.581301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7ea8b-9282-4e54-bb8d-84db43ec2d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:25.582865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ea8b-9282-4e54-bb8d-84db43ec2d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.583156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ea8b-9282-4e54-bb8d-84db43ec2d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.584474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7ea8b-9282-4e54-bb8d-84db43ec2d76 admin admin] PATH_INFO: `/v3/projects/a71da868d1874796b1910ec10bf8cd97/users/a5d45e5b20284dc292acc6259e596f45/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.591346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7ea8b-9282-4e54-bb8d-84db43ec2d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71da868d1874796b1910ec10bf8cd97, user_id=a5d45e5b20284dc292acc6259e596f45, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:25.596531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7ea8b-9282-4e54-bb8d-84db43ec2d76 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:25.600900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:25.602703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:25.641918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 811/1627] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:25 2022] PUT /identity/v3/projects/a71da868d1874796b1910ec10bf8cd97/users/a5d45e5b20284dc292acc6259e596f45/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:25.643438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1651594-8231-4d01-891b-f9c7a0722f17 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:25.643668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 817/1628] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:25.643787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1651594-8231-4d01-891b-f9c7a0722f17 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:25.643993 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1651594-8231-4d01-891b-f9c7a0722f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:25.691944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e1651594-8231-4d01-891b-f9c7a0722f17 None None] MFA Rules not processed for user `a5d45e5b20284dc292acc6259e596f45`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:25.693792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e1651594-8231-4d01-891b-f9c7a0722f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:25.879426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 812/1629] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:25.886066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:26.070953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fbb6c-9c6b-4f5d-9020-02c16e9f888c service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:26.116947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6a67-45cc-4f0a-bdff-fd1ca867e0de None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.117517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6a67-45cc-4f0a-bdff-fd1ca867e0de None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.117755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f6a67-45cc-4f0a-bdff-fd1ca867e0de None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.119281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 813/1630] 10.222.0.57 () {54 vars in 912 bytes} [Thu Sep 15 12:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:58:26.125347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.125639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.125915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.178346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:26.179863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:26.205710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641100e-7ebb-4ac3-bf3d-d65512600982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:26.206676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641100e-7ebb-4ac3-bf3d-d65512600982 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.207218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641100e-7ebb-4ac3-bf3d-d65512600982 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.207678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641100e-7ebb-4ac3-bf3d-d65512600982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.223378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:26.227540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641100e-7ebb-4ac3-bf3d-d65512600982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:26.230461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641100e-7ebb-4ac3-bf3d-d65512600982 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:26.242525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 818/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:26.245793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b78d-db80-4165-aed7-efdf38d6d081 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.246408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b78d-db80-4165-aed7-efdf38d6d081 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.246785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e00b78d-db80-4165-aed7-efdf38d6d081 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.254527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 819/1632] 10.222.0.57 () {54 vars in 912 bytes} [Thu Sep 15 12:58:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:58:26.256046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.256533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.256918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.288003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:26.299995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:26.340768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 814/1633] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 12:58:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:26.351031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:26.356698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:26.374514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b9579-d69a-4ebe-a219-100dd65c0019 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:26.383707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:26.384802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.386014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.387049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.418202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:26.426185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:26.460145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 815/1634] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:58:26.461561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:26.488832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10621e-f3ae-4130-b4b9-30c1d9ef8424 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:26.491627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 820/1635] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 12:58:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:26.508126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:26.513963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:26.514438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:26.515089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:26.601783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:26.605053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:26.640262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 816/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:27.085685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:27.105709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3f0a9-f2d3-41dd-ab1f-adcb54d5839e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:27.115930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09b45d-5a60-41d2-b34e-6a8ba3ce04aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:27.117012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09b45d-5a60-41d2-b34e-6a8ba3ce04aa service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:27.117012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09b45d-5a60-41d2-b34e-6a8ba3ce04aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:27.117267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09b45d-5a60-41d2-b34e-6a8ba3ce04aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:27.139637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09b45d-5a60-41d2-b34e-6a8ba3ce04aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:27.141134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09b45d-5a60-41d2-b34e-6a8ba3ce04aa service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:27.185304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 821/1637] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:27.640884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:27.815714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f871a8-182a-4d53-8470-995033e18f99 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:27.822308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:27.824222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:27.824553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:27.824832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] PATH_INFO: `/v3/users/204936c53844491682cef72ce215e035` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:27.826385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204936c53844491682cef72ce215e035)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:27.829590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:27.927319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:27.932966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204936c53844491682cef72ce215e035'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:27.949578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204936c53844491682cef72ce215e035'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:27.990822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204936c53844491682cef72ce215e035'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:28.040588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204936c53844491682cef72ce215e035'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:28.040588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204936c53844491682cef72ce215e035'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:28.057722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 817/1638] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:27 2022] DELETE /identity/v3/users/204936c53844491682cef72ce215e035 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:28.058234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:28.058662 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:28.059075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:28.110534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:28.117710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:28.269543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 822/1639] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:28.329337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:28.332914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:28.465085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbabcbb-a8bd-43a3-a073-97f2473f5c06 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:28.477969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0bce4-8b4f-4fac-b7f7-947478752fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:28.478801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0bce4-8b4f-4fac-b7f7-947478752fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:28.479286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0bce4-8b4f-4fac-b7f7-947478752fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:28.479579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0bce4-8b4f-4fac-b7f7-947478752fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:28.498491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0bce4-8b4f-4fac-b7f7-947478752fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:28.502317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0bce4-8b4f-4fac-b7f7-947478752fe4 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:28.529752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 818/1640] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:29.755211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:29.756728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:29.776588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b451-358d-4a35-af19-f0abae04a84c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:29.792306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:29.794051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:29.794660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:29.795103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] PATH_INFO: `/v3/projects/1741b69bb2be4ef08c76a4e0fb6e1487` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:29.799302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] RBAC: Authorizing `identity:delete_project(project_id=1741b69bb2be4ef08c76a4e0fb6e1487)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:29.801795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:29.928768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1741b69bb2be4ef08c76a4e0fb6e1487', 'request_id': 'req-1275acf0-30d6-4827-acfa-29defcbdddde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:29.946832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1741b69bb2be4ef08c76a4e0fb6e1487', 'request_id': 'req-1275acf0-30d6-4827-acfa-29defcbdddde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:29.964953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1741b69bb2be4ef08c76a4e0fb6e1487', 'request_id': 'req-1275acf0-30d6-4827-acfa-29defcbdddde'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:30.005711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] The token cache is being invalidate because project 1741b69bb2be4ef08c76a4e0fb6e1487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:30.006023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:30.007041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1275acf0-30d6-4827-acfa-29defcbdddde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:30.011834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 823/1641] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:29 2022] DELETE /identity/v3/projects/1741b69bb2be4ef08c76a4e0fb6e1487 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:30.017936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.018361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.018504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.071480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:30.073626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:30.245396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 819/1642] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:30.253424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.253780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.254131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.297963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:30.299080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:30.318793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 824/1643] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:30.323923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:30.340040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5026c-2098-4fd1-afbc-15d376c07bf1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:30.351871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:30.354379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.354379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.358988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.362182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:30.363492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:30.370892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:30.399936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 820/1644] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:30.404465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:30.418290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b474b-e9b1-4408-ab85-d1aea4b9bf8e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:30.434590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:30.435506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.436062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.436560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.437450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:30.443423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:30.493473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:30.509238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:30.516440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 825/1645] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:30 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:30.524500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb1efde-b49c-4b76-8010-1e8d1f0ef2d7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:30.659861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:30.686860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:30.687778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.688482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.689127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.690261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:30.692440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:30.731250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 821/1646] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:30 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:30.735237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:30.759567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0a8f5-04ff-4160-9954-4f00041ed87b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:30.769404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:30.770065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.770349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.770624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.772614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:30.774686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:30.819876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:30.820884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 826/1647] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:30.837868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fec378-aeeb-41b8-83ed-1134ea48a6d4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:30.846287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa48758-0bc9-4de6-97ab-202302ebce89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:30.847001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa48758-0bc9-4de6-97ab-202302ebce89 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.847353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa48758-0bc9-4de6-97ab-202302ebce89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.847703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa48758-0bc9-4de6-97ab-202302ebce89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.848457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa48758-0bc9-4de6-97ab-202302ebce89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:30.850153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa48758-0bc9-4de6-97ab-202302ebce89 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:30.886040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 822/1648] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:58:30.887331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:30.903040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05551905-db13-4473-9a27-ba6b8df99199 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:30.913851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:30.920577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:30.920926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:30.921474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] PATH_INFO: `/v3/projects/f0f182d731b1469a91370c6b92c0b62c/users/ad704038687143509f3bec687d220894/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:30.958463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f182d731b1469a91370c6b92c0b62c, user_id=ad704038687143509f3bec687d220894, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:30.962565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:30.998102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 827/1649] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:30 2022] PUT /identity/v3/projects/f0f182d731b1469a91370c6b92c0b62c/users/ad704038687143509f3bec687d220894/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:31.001519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:31.001975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:31.002394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:31.057819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] MFA Rules not processed for user `ad704038687143509f3bec687d220894`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:31.059861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:31.212179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 823/1650] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:31.214931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:31.384906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a95d2-04e6-4dd6-882d-878ab8e3ca75 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:31.405860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:31.409444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:31.410627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:31.411114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:31.413656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:31.415598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:31.456351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 828/1651] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:31 2022] POST /identity/v3/projects => generated 403 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:58:31.460440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:31.480329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9758e1b-fcef-4c4e-be4e-cedeb0fd028d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:31.687326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:31.687988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:31.688303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:31.688582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:31.689474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:31.691218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:31.775709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 824/1652] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:31 2022] POST /identity/v3/users => generated 308 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:31.779360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:31.797051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c150bad-6988-4b74-9112-c7d47ebf0d17 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:31.813762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:31.817378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:31.830282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:31.830925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:31.831278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:31.831512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:31.832137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:31.833769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:31.851801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab300e-d94f-4faf-859c-1f6854e55c96 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:31.862099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 829/1653] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:31.863529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:31.864541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:31.866499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:31.866826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:31.870109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] PATH_INFO: `/v3/users/21f78ba4b48249c688c3e3ed968d09d8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:31.871663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f78ba4b48249c688c3e3ed968d09d8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:31.875860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:31.882827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90fdf7-b3aa-4398-b597-85656f89fb24 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:31.890159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:31.892458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:31.892989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:31.893922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] PATH_INFO: `/v3/projects/547b00b520cb4a698a9331feb9af65f2/users/baea72c8b8bf4c12a9ce6a9464a22cc4/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:31.922469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547b00b520cb4a698a9331feb9af65f2, user_id=baea72c8b8bf4c12a9ce6a9464a22cc4, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:31.927331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:31.958202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 830/1654] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:31 2022] PUT /identity/v3/projects/547b00b520cb4a698a9331feb9af65f2/users/baea72c8b8bf4c12a9ce6a9464a22cc4/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:31.959526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:31.996109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:32.014520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f78ba4b48249c688c3e3ed968d09d8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:32.031991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f78ba4b48249c688c3e3ed968d09d8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:32.064005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f78ba4b48249c688c3e3ed968d09d8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:32.067797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f78ba4b48249c688c3e3ed968d09d8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:32.090639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-69243ab9-1ede-4d4f-87e0-f12cd4c28f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f78ba4b48249c688c3e3ed968d09d8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:32.111557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 825/1655] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:31 2022] DELETE /identity/v3/users/21f78ba4b48249c688c3e3ed968d09d8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:32.116353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:32.116652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:32.116933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:32.177617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:32.178866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:32.206375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a30d3-d904-47ea-8e24-1e8321758144 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:32.269596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:32.363562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 826/1656] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:32.425246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd5315-cb65-4149-a9a4-c528bfda2236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:32.425996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd5315-cb65-4149-a9a4-c528bfda2236 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:32.426219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd5315-cb65-4149-a9a4-c528bfda2236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:32.426435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd5315-cb65-4149-a9a4-c528bfda2236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:32.427091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd5315-cb65-4149-a9a4-c528bfda2236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:32.428791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd5315-cb65-4149-a9a4-c528bfda2236 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:32.474208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 831/1657] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:32.476219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:32.479123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:32.507309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2720db-402e-473c-a9c6-3900aaf8b68f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:32.520440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:32.521226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:32.521579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:32.521927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] PATH_INFO: `/v3/projects/547b00b520cb4a698a9331feb9af65f2/users/baea72c8b8bf4c12a9ce6a9464a22cc4/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:32.524114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=547b00b520cb4a698a9331feb9af65f2, user_id=baea72c8b8bf4c12a9ce6a9464a22cc4, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:32.527765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:32.569982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 827/1658] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:32 2022] PUT /identity/v3/projects/547b00b520cb4a698a9331feb9af65f2/users/baea72c8b8bf4c12a9ce6a9464a22cc4/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:32.581065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:32.581436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:32.581784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:32.626012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] MFA Rules not processed for user `baea72c8b8bf4c12a9ce6a9464a22cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:32.627433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:32.719976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:32.721783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:32.840885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 832/1659] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:32.859164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:32.861054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:32.944183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f822145-a182-48e4-a410-261506795a6b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:32.955661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:32.959834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:32.960140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:32.960452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:33.063762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4a26e-60be-4c38-8386-aa36ef5e7b6a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:33.105007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49cd7d-fc16-4af6-9c31-408e79e17171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:33.105862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49cd7d-fc16-4af6-9c31-408e79e17171 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:33.106146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49cd7d-fc16-4af6-9c31-408e79e17171 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:33.107459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49cd7d-fc16-4af6-9c31-408e79e17171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:33.142052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49cd7d-fc16-4af6-9c31-408e79e17171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:33.143930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49cd7d-fc16-4af6-9c31-408e79e17171 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:33.162555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 833/1660] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:58:33.177287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:33.179460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:33.184501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:33.211430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 828/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:58:35.254135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:35.255187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:35.255693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:35.335552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] MFA Rules not processed for user `cd22d2df9ced4bb0961596385854422d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:35.339219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:35.357781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:35.359817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:35.376633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4236d-907a-46ba-b9d1-860d9dc46bcc service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:35.383824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:35.385574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:35.386125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:35.386640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] PATH_INFO: `/v3/projects/29549a86d26043369f989b70d85596d2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:35.390115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=29549a86d26043369f989b70d85596d2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:35.394367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:58:35.394367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 15 12:58:35.394367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: Sep 15 12:58:35.396303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:35.437483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:35.524981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 834/1662] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:35.532081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:35.534255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:35.544354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29549a86d26043369f989b70d85596d2', 'request_id': 'req-26e425b5-45c7-468b-bbfe-63e527291c7d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:35.564789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29549a86d26043369f989b70d85596d2', 'request_id': 'req-26e425b5-45c7-468b-bbfe-63e527291c7d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:35.597209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29549a86d26043369f989b70d85596d2', 'request_id': 'req-26e425b5-45c7-468b-bbfe-63e527291c7d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:35.639474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] The token cache is being invalidate because project 29549a86d26043369f989b70d85596d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:35.640974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:35.645847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-26e425b5-45c7-468b-bbfe-63e527291c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:35.656208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 829/1663] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:35 2022] DELETE /identity/v3/projects/29549a86d26043369f989b70d85596d2 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:35.659180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff7b30-bfb8-4069-9af8-dda04e9616a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:35.659503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff7b30-bfb8-4069-9af8-dda04e9616a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:35.659884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff7b30-bfb8-4069-9af8-dda04e9616a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:35.733424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493389c-576c-4323-a9ac-1d252fd1353e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:35.742118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-72ff7b30-bfb8-4069-9af8-dda04e9616a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:35.753215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-72ff7b30-bfb8-4069-9af8-dda04e9616a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:35.867758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:35.868424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:35.868810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:35.869137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:35.872582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:35.923343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 830/1664] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:35.930862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:35.931307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:35.932867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:35.958508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:35.959928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:35.996499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 831/1665] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:36.001381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.022287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d82f3f-58dc-48f4-8b79-f7a40e4166f0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.024673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.027499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.036086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.038757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.040621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.041166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.042916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:36.044596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.046580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.049034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 835/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:58:36.090959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 832/1667] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:36.097298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.119210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1b032-c9f2-41e3-a7c5-fa6b7577ae52 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.131279 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.131994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.132311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.132578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.133233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.135744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.153072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:36.202866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 836/1668] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:36 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:36.207324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.223329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64da313-63f4-4399-bbda-69c18795a7d9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.374705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.376724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.377156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.377372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.378122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.386695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.405560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:36.435779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 833/1669] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:36 2022] POST /identity/v3/users => generated 284 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:36.440400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.469376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cf24f-7218-400b-bf4b-ca82c4578554 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.485855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.488663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.488957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.491157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.491948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.500016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.526864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 837/1670] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:36.532650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.555936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bb08-b98a-4dd1-8b35-599ebbfc13ed admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.571156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.571930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.573164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.573592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.575901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.581629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.605884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 834/1671] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:58:36.610142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.623635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b981-12f4-49f7-bb49-be70509dbd24 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.639475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c8313-e02e-4b34-a174-0d4245c9cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.640244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c8313-e02e-4b34-a174-0d4245c9cf65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.640564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c8313-e02e-4b34-a174-0d4245c9cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.640845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c8313-e02e-4b34-a174-0d4245c9cf65 admin admin] PATH_INFO: `/v3/projects/f228dcd0990f4d33a8c145933b572005/users/3a32bdcd6f154d83a508920b546a86de/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.686373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c8313-e02e-4b34-a174-0d4245c9cf65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f228dcd0990f4d33a8c145933b572005, user_id=3a32bdcd6f154d83a508920b546a86de, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.689234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c8313-e02e-4b34-a174-0d4245c9cf65 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:36.708651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:36.712601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:36.738761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 838/1672] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:36 2022] PUT /identity/v3/projects/f228dcd0990f4d33a8c145933b572005/users/3a32bdcd6f154d83a508920b546a86de/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:36.757283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.761413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.762078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.782254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41006-29cb-4bb1-abe9-78f8f96deff0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:36.879474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] MFA Rules not processed for user `3a32bdcd6f154d83a508920b546a86de`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:36.883074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:36.987498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:36.987954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:36.988238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:36.988599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] PATH_INFO: `/v3/users/ad704038687143509f3bec687d220894` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:36.993074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad704038687143509f3bec687d220894)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:36.997363 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:37.109696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 839/1673] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:37.118540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:37.120655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:37.177100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad704038687143509f3bec687d220894'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:37.206509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad704038687143509f3bec687d220894'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:37.274603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad704038687143509f3bec687d220894'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:37.276720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad704038687143509f3bec687d220894'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:37.312045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ee47e7d1-e472-4516-b515-336af21b1f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad704038687143509f3bec687d220894'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:37.337618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 835/1674] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:36 2022] DELETE /identity/v3/users/ad704038687143509f3bec687d220894 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:58:37.344441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:37.348369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:37.348686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:37.430032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e65f64-5dd0-4a77-94da-c07ebf0257eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:37.431822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:37.433761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:37.600338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:37.601102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:37.601282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:37.601571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:37.615679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:37.620842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:37.640507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:37.647574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 836/1675] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:37.702413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:37.704014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:37.795720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:37.801672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51347fec-78f9-4e12-972d-23c4dd517d97 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:37.851041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 840/1676] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:37.899874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f845d89-aab7-4461-8bfd-a91e48a5f3cf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:37.917174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:37.918314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:37.918314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:37.918495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:37.942718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:37.944216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:37.991626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 837/1677] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:39.705450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:39.705980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:39.706209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:39.828839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] MFA Rules not processed for user `cff1145566f1408a9a22da9088581024`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:39.830108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:40.103245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:40.105299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:40.108577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 841/1678] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:40.174670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:40.176121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:40.215040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c38d71-33fe-4bbd-8520-d170796dcba6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:40.240875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:40.242148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:40.242148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:40.242330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] PATH_INFO: `/v3/projects/f0f182d731b1469a91370c6b92c0b62c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:40.245939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f182d731b1469a91370c6b92c0b62c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:40.248113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:40.341239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8348c-46ae-49b1-892c-0aeb33b7e2f6 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:40.368392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:40.369073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:40.369358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:40.369647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:40.419674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:40.426170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:40.469611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 838/1679] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:40.540589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f182d731b1469a91370c6b92c0b62c', 'request_id': 'req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:40.600744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f182d731b1469a91370c6b92c0b62c', 'request_id': 'req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:40.633032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f182d731b1469a91370c6b92c0b62c', 'request_id': 'req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:40.668092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] The token cache is being invalidate because project f0f182d731b1469a91370c6b92c0b62c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:40.668538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:40.671857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34d4c3cd-c688-45e5-833d-a1bb6f6d23d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:40.677657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 842/1680] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:40 2022] DELETE /identity/v3/projects/f0f182d731b1469a91370c6b92c0b62c => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:40.679267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:40.703995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1fdae-ee3f-4284-b96b-2c7cc763fdcf service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:40.934683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:40.935757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:40.936245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:40.936792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] PATH_INFO: `/v3/users/baea72c8b8bf4c12a9ce6a9464a22cc4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:40.938495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baea72c8b8bf4c12a9ce6a9464a22cc4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:40.941334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:41.018649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:41.038998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baea72c8b8bf4c12a9ce6a9464a22cc4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:41.061912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baea72c8b8bf4c12a9ce6a9464a22cc4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:41.115662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baea72c8b8bf4c12a9ce6a9464a22cc4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:41.117207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baea72c8b8bf4c12a9ce6a9464a22cc4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:41.136449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1eeb3314-b91b-4199-b14f-902f7b22e708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baea72c8b8bf4c12a9ce6a9464a22cc4'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:41.147309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 839/1681] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:40 2022] DELETE /identity/v3/users/baea72c8b8bf4c12a9ce6a9464a22cc4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:58:42.273881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:42.275045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:42.275766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:42.358077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] MFA Rules not processed for user `082e5aa56ac147e6a8f8c797b269ff23`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:42.360653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:42.404549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:42.542918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 843/1682] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:42.558812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:42.559357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:42.559357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:42.622624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] MFA Rules not processed for user `082e5aa56ac147e6a8f8c797b269ff23`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:42.625321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:42.636812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 840/1683] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:42.655307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:42.657368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:42.884069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3b9e7-7c93-4c2b-9fc6-1433c44497e1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:42.899133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:42.900251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a70aa-94af-4b6e-a80f-fdf622efb0db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:42.901837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a70aa-94af-4b6e-a80f-fdf622efb0db service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:42.902332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a70aa-94af-4b6e-a80f-fdf622efb0db service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:42.902754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a70aa-94af-4b6e-a80f-fdf622efb0db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:42.903643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:42.914133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a70aa-94af-4b6e-a80f-fdf622efb0db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:42.918083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a70aa-94af-4b6e-a80f-fdf622efb0db service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:42.957112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 844/1684] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:43.157889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7bb20-a0ee-423d-94b4-ff5cac1b6022 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:43.172843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:43.175310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.177210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.177668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] PATH_INFO: `/v3/users/27f02bbd8abd45aaa3d5fd6070b5adc1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.181315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f02bbd8abd45aaa3d5fd6070b5adc1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:43.186202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:43.314385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f02bbd8abd45aaa3d5fd6070b5adc1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.329261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f02bbd8abd45aaa3d5fd6070b5adc1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.355278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f02bbd8abd45aaa3d5fd6070b5adc1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.356205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f02bbd8abd45aaa3d5fd6070b5adc1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.371407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f02bbd8abd45aaa3d5fd6070b5adc1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.379840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 841/1685] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:42 2022] DELETE /identity/v3/users/27f02bbd8abd45aaa3d5fd6070b5adc1 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:43.384377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.385616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.386333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.417118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:43.418724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:43.440247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:43.442079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:43.494620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:43.616306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4f76d-b5c8-4644-84ce-410f43877f62 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:43.622712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 845/1686] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:43.634212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:43.635136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.635520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.636032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] PATH_INFO: `/v3/projects/547b00b520cb4a698a9331feb9af65f2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.639085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547b00b520cb4a698a9331feb9af65f2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:43.642244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:43.664412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:43.667519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:43.724216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547b00b520cb4a698a9331feb9af65f2', 'request_id': 'req-90a943f2-3758-41b7-a947-d47e91911073'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.732813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547b00b520cb4a698a9331feb9af65f2', 'request_id': 'req-90a943f2-3758-41b7-a947-d47e91911073'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.740764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547b00b520cb4a698a9331feb9af65f2', 'request_id': 'req-90a943f2-3758-41b7-a947-d47e91911073'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.769219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] The token cache is being invalidate because project 547b00b520cb4a698a9331feb9af65f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:43.770309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.771568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-90a943f2-3758-41b7-a947-d47e91911073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:43.773857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 842/1687] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:43 2022] DELETE /identity/v3/projects/547b00b520cb4a698a9331feb9af65f2 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:43.775719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae149d-88d1-45d6-8e33-503b20dd79fa None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.776355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae149d-88d1-45d6-8e33-503b20dd79fa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.777296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae149d-88d1-45d6-8e33-503b20dd79fa None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.780039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 843/1688] 10.222.0.57 () {54 vars in 912 bytes} [Thu Sep 15 12:58:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:58:43.792491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da8923-5827-415e-a679-3552a60c7bac None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.793285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da8923-5827-415e-a679-3552a60c7bac None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.793829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da8923-5827-415e-a679-3552a60c7bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.839480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-80da8923-5827-415e-a679-3552a60c7bac None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:43.842316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-80da8923-5827-415e-a679-3552a60c7bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:43.843169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:43.865265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19d0e-a964-4c27-81d7-f2193626b62e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:43.930235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 844/1689] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:43.934198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070d126-2721-4551-a85f-29a8d1257e61 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.935310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070d126-2721-4551-a85f-29a8d1257e61 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.935704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070d126-2721-4551-a85f-29a8d1257e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.985385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:43.985385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:43.985737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:43.986412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:43.990940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:44.000656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1070d126-2721-4551-a85f-29a8d1257e61 None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:44.001970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1070d126-2721-4551-a85f-29a8d1257e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:44.009642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 845/1690] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:44.012125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.012252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.012621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.042174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] MFA Rules not processed for user `cd22d2df9ced4bb0961596385854422d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:44.044480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.046369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:44.047247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f8088-64d1-46bd-9dde-90cda6ba8707 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.072389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 846/1691] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:44.076012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.089099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.089695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.139618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:44.140954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:44.151452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 847/1692] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:44.152436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.153938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:44.161350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:44.174378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309c69-365b-4761-9e70-e2c0df569fd9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.184114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.184768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.187400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.188507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.207214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.210444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.238473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 848/1693] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:44.238473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.238759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 846/1694] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:44.254408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f3a0f-df02-474e-a568-2018208407ee service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.259764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.260707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.261190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.261619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.262412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:44.262867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.264720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.284447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 849/1695] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:44.289036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.312666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0316f-34f2-45ea-b6b9-e4544bb8704f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.323027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.325680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.326113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.326395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.329152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.336038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.398470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 847/1696] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:44 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:44.402754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.413380 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c291-6c9b-4940-8735-76919b3b4bbd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.569900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.570539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.570822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.571097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.571743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.575872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.624108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.638000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 850/1697] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:44 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:44.643524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805fbee-fa8e-44e1-9c02-775cb3df7bf8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.656253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.656925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.657204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.657538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.658196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.659820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.676790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.677752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 848/1698] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:44.699103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab3ff4-023e-4456-9871-c5e6a85c037d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.708540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460d374-e40b-4c90-9e23-d996b01fcdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.709165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460d374-e40b-4c90-9e23-d996b01fcdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.709462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460d374-e40b-4c90-9e23-d996b01fcdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.709726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460d374-e40b-4c90-9e23-d996b01fcdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.710563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460d374-e40b-4c90-9e23-d996b01fcdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.712013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460d374-e40b-4c90-9e23-d996b01fcdb4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.751976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:44.760066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 851/1699] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:44.765899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874406c-72c8-406b-96b8-cd99ac515313 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:44.779383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:44.785667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.787317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.787916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] PATH_INFO: `/v3/projects/61a38a268b9547ad86f9a7bfca700fb3/users/adb0bead59604e0592c7e12ec3e9409a/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.815078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a38a268b9547ad86f9a7bfca700fb3, user_id=adb0bead59604e0592c7e12ec3e9409a, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:44.819100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:44.880740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 849/1700] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:44 2022] PUT /identity/v3/projects/61a38a268b9547ad86f9a7bfca700fb3/users/adb0bead59604e0592c7e12ec3e9409a/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:44.885625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:44.886028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:44.886415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:44.943175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] MFA Rules not processed for user `adb0bead59604e0592c7e12ec3e9409a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:44.946043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:45.007181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:45.076321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 852/1701] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:45.081041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:45.290096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fcc12-8a93-4f65-b162-34bb2df0c634 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:45.308085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:45.309768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:45.310284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:45.310810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:45.312262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:45.314742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:45.379818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 850/1702] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:45 2022] POST /identity/v3/projects => generated 379 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:58:45.383865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:45.404683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970986dc-d0ac-4adb-8af1-048582bc18d7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:45.593771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:45.594608 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:45.594881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:45.595290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:45.598143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:45.600302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:45.649004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 853/1703] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:45 2022] POST /identity/v3/users => generated 296 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:45.653813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:45.669911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107be511-26a1-4377-9a29-bd3ddecf009d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:45.682499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:45.683931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:45.685742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:45.686309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:45.691789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:45.695959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:45.737033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:45.737580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 851/1704] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:45.757915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568789-6ed9-49b2-af05-88ff40c5d093 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:45.795639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:45.800424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:45.800424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:45.800424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] PATH_INFO: `/v3/projects/9f29ae38596f499d83c4f2a5c503440a/users/1824f20481264f32b81b9008d6de134e/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:45.850854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f29ae38596f499d83c4f2a5c503440a, user_id=1824f20481264f32b81b9008d6de134e, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:45.853972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:45.894772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 854/1705] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:45 2022] PUT /identity/v3/projects/9f29ae38596f499d83c4f2a5c503440a/users/1824f20481264f32b81b9008d6de134e/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:45.895953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:45.918715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006f5b4-f658-41d1-83e7-a2b1e78d0a06 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:45.939183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:45.940971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.119818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:46.140019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:46.140869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.141020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.141220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.142133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:46.143760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:46.148017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b30bf8-69f3-4c90-aabf-19a5a8f635ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:46.173365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 852/1706] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:46.176640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:46.179032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:46.179709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.180088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.180393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] PATH_INFO: `/v3/projects/de365ed54c494a3da402b036ca326abc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.185221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.187383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=de365ed54c494a3da402b036ca326abc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:46.189356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:46.291923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb525-530c-4a4c-8934-9fba392642a1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:46.300976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:46.301619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.302064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.302254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.304697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.321261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:46.324945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb955137-0eba-4ec2-b32f-ed976bcda434 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:46.342215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 853/1707] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:46.344746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.345038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.345441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.364411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:46.377873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de365ed54c494a3da402b036ca326abc', 'request_id': 'req-e27b4cc5-95a8-4392-b540-9778e4e6dbff'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:46.393697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de365ed54c494a3da402b036ca326abc', 'request_id': 'req-e27b4cc5-95a8-4392-b540-9778e4e6dbff'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:46.403678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] MFA Rules not processed for user `e2ef1277bf944722aa33077985fb4734`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:46.405039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.407621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de365ed54c494a3da402b036ca326abc', 'request_id': 'req-e27b4cc5-95a8-4392-b540-9778e4e6dbff'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:46.447793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] The token cache is being invalidate because project de365ed54c494a3da402b036ca326abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:46.448079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:46.452713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:46.467017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 855/1708] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:45 2022] DELETE /identity/v3/projects/de365ed54c494a3da402b036ca326abc => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:46.468027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:46.472841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.582134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 854/1709] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:46.583199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:46.585328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.658503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b4cc5-95a8-4392-b540-9778e4e6dbff admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:46.663864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a917a6-d35e-4545-a9ae-1c35db6acb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:46.664750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a917a6-d35e-4545-a9ae-1c35db6acb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.665295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a917a6-d35e-4545-a9ae-1c35db6acb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.665672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a917a6-d35e-4545-a9ae-1c35db6acb3f admin admin] PATH_INFO: `/v3/projects/9f29ae38596f499d83c4f2a5c503440a/users/1824f20481264f32b81b9008d6de134e/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.668711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a917a6-d35e-4545-a9ae-1c35db6acb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f29ae38596f499d83c4f2a5c503440a, user_id=1824f20481264f32b81b9008d6de134e, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:46.671697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a917a6-d35e-4545-a9ae-1c35db6acb3f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:46.724702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 856/1710] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:46 2022] PUT /identity/v3/projects/9f29ae38596f499d83c4f2a5c503440a/users/1824f20481264f32b81b9008d6de134e/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:46.732159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d748b82-d6d9-4b06-b0b2-51b3e68a33eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.732594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d748b82-d6d9-4b06-b0b2-51b3e68a33eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.732729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d748b82-d6d9-4b06-b0b2-51b3e68a33eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.800764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1d748b82-d6d9-4b06-b0b2-51b3e68a33eb None None] MFA Rules not processed for user `2f0feb7134ee408b989b398a315850fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:46.802407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1d748b82-d6d9-4b06-b0b2-51b3e68a33eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:46.823404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54400135-7151-4a82-8a24-85db35d94c31 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:46.872001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:46.902228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 857/1711] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:46.913223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294c7fc-b61e-4b59-823b-991758e4d43a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.913742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294c7fc-b61e-4b59-823b-991758e4d43a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.914270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294c7fc-b61e-4b59-823b-991758e4d43a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.916338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 858/1712] 10.222.0.57 () {54 vars in 912 bytes} [Thu Sep 15 12:58:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:58:46.917974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f32b82-e498-43bc-b807-6e93703b3da4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:46.918378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f32b82-e498-43bc-b807-6e93703b3da4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:46.918727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f32b82-e498-43bc-b807-6e93703b3da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:46.944058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-19f32b82-e498-43bc-b807-6e93703b3da4 None None] MFA Rules not processed for user `e2ef1277bf944722aa33077985fb4734`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:46.946308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-19f32b82-e498-43bc-b807-6e93703b3da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.040828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:47.057942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 859/1713] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:47.059811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c34c-8153-45da-8643-49a6fc7d6cf1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.069131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:47.069796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.070078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.070890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] PATH_INFO: `/v3/users/a5d45e5b20284dc292acc6259e596f45` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.073494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c34c-8153-45da-8643-49a6fc7d6cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.074003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d45e5b20284dc292acc6259e596f45)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:47.075966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:47.076387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0c34c-8153-45da-8643-49a6fc7d6cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.144988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-83a0c34c-8153-45da-8643-49a6fc7d6cf1 None None] MFA Rules not processed for user `1824f20481264f32b81b9008d6de134e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:47.146687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-83a0c34c-8153-45da-8643-49a6fc7d6cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.190267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d45e5b20284dc292acc6259e596f45'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:47.206099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d45e5b20284dc292acc6259e596f45'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:47.232932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d45e5b20284dc292acc6259e596f45'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:47.233946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d45e5b20284dc292acc6259e596f45'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:47.250759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3e534e9f-ba5e-4b94-8186-1a9fe00f86e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d45e5b20284dc292acc6259e596f45'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:47.263398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 855/1714] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:46 2022] DELETE /identity/v3/users/a5d45e5b20284dc292acc6259e596f45 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:47.281914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.283762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.286296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.296674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 860/1715] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:47.298323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.298790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.299047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.325057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] MFA Rules not processed for user `cff1145566f1408a9a22da9088581024`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:47.326430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.363471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:47.367851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.478809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 861/1716] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:47.489464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:47.500892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.538218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 856/1717] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 12:58:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:47.547251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:47.550795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.635416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe786fe-841b-4277-ba49-2fb4df03da62 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:47.642683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:47.643469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.643755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.644025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.646493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.701253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:47.733931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:47.758655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c3c05-5b3c-4e7c-ad7f-872e94dca73c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:47.764416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:47.770139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:47.780805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:47.781953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.783019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.785188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 862/1718] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:47.786203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:47.787880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:47.795739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.826679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e5d91-5250-46bf-881e-c8f7b9f915de service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:47.831237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:47.832148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:47.832266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:47.832670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:47.848668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:47.850469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:47.866831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 863/1719] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:47.965477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:47.966930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:47.981021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 857/1720] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:48.100478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:48.102238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:48.185895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:48.197007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312089-ff69-4f57-8532-39e5a5979cc2 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:48.204965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:48.205949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:48.206988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:48.207385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:48.209937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:48.287759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137918a-bad7-4751-81a3-ac88aaab1d32 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:48.302777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:48.303744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:48.304627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:48.305063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] PATH_INFO: `/v3/projects/a71da868d1874796b1910ec10bf8cd97` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:48.308393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71da868d1874796b1910ec10bf8cd97)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:48.314375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:48.317829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:48.321377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:48.345770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 858/1721] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:58:48.359116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:48.410202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944757-829c-4607-bc48-ddedba235f20 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:48.434431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:48.436245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:48.436552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:48.437909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:48.443585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:48.484503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71da868d1874796b1910ec10bf8cd97', 'request_id': 'req-ec74baa1-37fa-4a21-8e67-8e495e13f699'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:48.507290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71da868d1874796b1910ec10bf8cd97', 'request_id': 'req-ec74baa1-37fa-4a21-8e67-8e495e13f699'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:48.529076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71da868d1874796b1910ec10bf8cd97', 'request_id': 'req-ec74baa1-37fa-4a21-8e67-8e495e13f699'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:48.563131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] The token cache is being invalidate because project a71da868d1874796b1910ec10bf8cd97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:48.563427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:48.569054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ec74baa1-37fa-4a21-8e67-8e495e13f699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:48.577390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 864/1722] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:48 2022] DELETE /identity/v3/projects/a71da868d1874796b1910ec10bf8cd97 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:48.579586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:48.579887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:48.580155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:48.692419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:48.705541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:48.706591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:48.713571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532d3f3-b867-4880-8c41-fc4e43ede0ee service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:48.766727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 859/1723] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:48.855014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:48.945768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:48.946192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:48.946475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:48.949248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 865/1724] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:48.985325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:48.987210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:48.998034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 860/1725] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:49.006974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:49.021288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325a77-9be3-4849-aab6-8178fa7635c1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:49.029016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:49.029974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.030415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.030873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.031864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:49.035402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:49.037503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:49.054237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 866/1726] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:49.059013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:49.078242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ef200-62ad-4f48-b412-d2af6b1af1f3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:49.093512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:49.094634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.094920 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.096324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.098327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:49.103904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:49.195388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 861/1727] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:49 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:49.196674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:49.216829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbcc8a4-b2b9-44d4-9703-feb836bd3acc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:49.354067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:49.355322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.355616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.355938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.358739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:49.363525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:49.412631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 867/1728] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:49 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:58:49.417089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:49.422858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:49.430583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c06ef-6b01-4322-9dec-3096b2b62fc6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:49.444472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:49.445271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.445565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.445886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.446553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:49.449307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:49.481256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 862/1729] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:49.484527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:49.504907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e811-75b3-4ef1-a765-c98b5a8cd1e8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:49.524881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9d87b-2ad6-486b-a279-b4ccaae39aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:49.526267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9d87b-2ad6-486b-a279-b4ccaae39aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.526560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9d87b-2ad6-486b-a279-b4ccaae39aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.526978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9d87b-2ad6-486b-a279-b4ccaae39aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.528778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9d87b-2ad6-486b-a279-b4ccaae39aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:49.531198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9d87b-2ad6-486b-a279-b4ccaae39aa1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:49.564924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 868/1730] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:49.567709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:49.589639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8392d-0c77-42a7-9e3a-bb313ba908ae admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:49.612300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:49.615921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.617674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.618096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] PATH_INFO: `/v3/projects/aba2cec813434bd2a1e3adb2fc6d7bc7/users/e99968a1ddd7416bb0b736c6b914aa1e/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.668416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba2cec813434bd2a1e3adb2fc6d7bc7, user_id=e99968a1ddd7416bb0b736c6b914aa1e, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:49.677797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:49.725328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 863/1731] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:49 2022] PUT /identity/v3/projects/aba2cec813434bd2a1e3adb2fc6d7bc7/users/e99968a1ddd7416bb0b736c6b914aa1e/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:49.746191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:49.751049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:49.753898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:49.862323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] MFA Rules not processed for user `e99968a1ddd7416bb0b736c6b914aa1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:49.864184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:49.978036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 869/1732] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:49.982303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:50.150752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3782a06-2be6-40e4-81c3-5bd973876986 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:50.165952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859f33e-a957-4108-ae1d-61c780d460ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:50.166291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f33e-a957-4108-ae1d-61c780d460ce admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:50.166602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f33e-a957-4108-ae1d-61c780d460ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:50.166847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f33e-a957-4108-ae1d-61c780d460ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:50.167838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f33e-a957-4108-ae1d-61c780d460ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:50.171428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f33e-a957-4108-ae1d-61c780d460ce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:50.228494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 864/1733] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:49 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:50.232617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:50.247901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:50.248211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:50.248486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:50.253721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7171e6a-150d-426e-a128-ebaf7cf1a79f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:50.358698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:50.436378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:50.437027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:50.437313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:50.437589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:50.438227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:50.443086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:50.471925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:50.473677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:50.502240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 870/1734] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:58:50 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:58:50.506004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:50.552144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512764ee-8151-4381-b5d2-a54fcd479752 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:50.577171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:50.577825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:50.578231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:50.578395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:50.579080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:50.580680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:50.608197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 865/1735] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 12:58:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:50.652211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:50.654434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 871/1736] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:50.710115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e5b4c-4f1c-46e3-996f-20d6b2f12cb7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:50.724974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:50.725660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:50.725944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:50.726223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] PATH_INFO: `/v3/projects/4f82f3da3aa941ceb6446053cf60a840/users/de43eb03946a4b4b9585e79860424b65/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:50.808232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f82f3da3aa941ceb6446053cf60a840, user_id=de43eb03946a4b4b9585e79860424b65, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:50.813304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:50.842085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:50.861374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 866/1737] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:50 2022] PUT /identity/v3/projects/4f82f3da3aa941ceb6446053cf60a840/users/de43eb03946a4b4b9585e79860424b65/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:50.865168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:50.888244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fd341-46a0-4444-a69f-0501630f286f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:50.994892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:50.996696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:51.112768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:51.113415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:51.113700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:51.113972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:51.117427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:51.126621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:51.157319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 872/1738] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:58:51.159946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:51.195616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296cdb1-1709-4675-8b80-17a43b82aaf5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:51.209205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b5a8-769b-4292-9f1c-065f9455f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:51.210071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b5a8-769b-4292-9f1c-065f9455f6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:51.210400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b5a8-769b-4292-9f1c-065f9455f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:51.210698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b5a8-769b-4292-9f1c-065f9455f6fd admin admin] PATH_INFO: `/v3/projects/4f82f3da3aa941ceb6446053cf60a840/users/de43eb03946a4b4b9585e79860424b65/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:51.214360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b5a8-769b-4292-9f1c-065f9455f6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f82f3da3aa941ceb6446053cf60a840, user_id=de43eb03946a4b4b9585e79860424b65, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:51.221346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b5a8-769b-4292-9f1c-065f9455f6fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:51.227868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8646335-8d59-41ec-98b5-c9872bd83b2a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:51.247224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:51.248371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:51.250375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:51.251254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:51.252753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 873/1739] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:58:51 2022] PUT /identity/v3/projects/4f82f3da3aa941ceb6446053cf60a840/users/de43eb03946a4b4b9585e79860424b65/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:51.264845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77db2-7b82-475f-bb97-8438f4901feb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:51.265195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77db2-7b82-475f-bb97-8438f4901feb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:51.265478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de77db2-7b82-475f-bb97-8438f4901feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:51.343387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2de77db2-7b82-475f-bb97-8438f4901feb None None] MFA Rules not processed for user `de43eb03946a4b4b9585e79860424b65`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:51.347043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2de77db2-7b82-475f-bb97-8438f4901feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:51.384215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:51.386612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:51.452371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 867/1740] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 12:58:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:58:51.545905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 874/1741] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:51.559610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:51.562591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:51.799815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068225-f541-4a80-be7a-eb41e74de4e4 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:51.940172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:51.993782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:51.994434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:51.994751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:51.997765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:52.046179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:52.047678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:52.131637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 868/1742] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:53.042386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:53.042969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:53.042969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:53.099462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] MFA Rules not processed for user `e2ef1277bf944722aa33077985fb4734`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:53.107959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:53.233074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:53.424112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 875/1743] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:53.623248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:53.641905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c54ef-2941-483a-a833-a3baeb8a1c00 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:53.672765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de43eb03946a4b4b9585e79860424b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f82f3da3aa941ceb6446053cf60a840', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:53.673391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:53.680193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:53.680547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:53.689347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:53.694522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:53.703677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:53.753719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 869/1744] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:53.758167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:53.816723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848c30e-8894-476c-b13a-02619ed973ae None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:53.831298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de43eb03946a4b4b9585e79860424b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f82f3da3aa941ceb6446053cf60a840', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:53.833281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:53.836469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:53.836980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:53.843360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:53.849636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:53.894715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 876/1745] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:58:53 2022] POST /identity/v3/projects => generated 354 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:58:53.898707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:53.917470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590cdc-65b4-4eb0-84ea-06333c11558a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:54.075295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de43eb03946a4b4b9585e79860424b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f82f3da3aa941ceb6446053cf60a840', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:54.076002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:54.076398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:54.076865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:54.077515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:58:54.077973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:54.079720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:54.119528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 870/1746] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:58:54.167047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:54.180138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320a969-47e9-4a1b-98f1-3ffed48a343a tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:54.187010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de43eb03946a4b4b9585e79860424b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f82f3da3aa941ceb6446053cf60a840', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:54.187943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:54.188422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:54.188858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] PATH_INFO: `/v3/projects/e2f6d6267c64457785c92d3de958658d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:54.191397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e2f6d6267c64457785c92d3de958658d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:54.193862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:54.256795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f6d6267c64457785c92d3de958658d', 'request_id': 'req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.263806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f6d6267c64457785c92d3de958658d', 'request_id': 'req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.272048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f6d6267c64457785c92d3de958658d', 'request_id': 'req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.292917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] The token cache is being invalidate because project e2f6d6267c64457785c92d3de958658d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:54.293811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.294240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a8e0bda4-91fb-438c-8cfa-2b697b2aab9c tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.296861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 877/1747] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:54 2022] DELETE /identity/v3/projects/e2f6d6267c64457785c92d3de958658d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:54.570258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:54.573141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:54.615055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80790bf-6934-4b58-9d29-da14278a4b25 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:54.615339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80790bf-6934-4b58-9d29-da14278a4b25 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:54.615807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80790bf-6934-4b58-9d29-da14278a4b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:54.651864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a80790bf-6934-4b58-9d29-da14278a4b25 None None] MFA Rules not processed for user `2f0feb7134ee408b989b398a315850fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:54.653173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a80790bf-6934-4b58-9d29-da14278a4b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:54.674120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0ed9e-f125-4056-87d1-547c28b48250 tempest-VolumeQuotaClassesTest-1780331411 tempest-VolumeQuotaClassesTest-1780331411-project-admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:54.767665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:54.768138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:54.768760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:54.769222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] PATH_INFO: `/v3/users/e99968a1ddd7416bb0b736c6b914aa1e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:54.770599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99968a1ddd7416bb0b736c6b914aa1e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:54.773223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:54.789195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:54.805814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 878/1748] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:54.877500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99968a1ddd7416bb0b736c6b914aa1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.886290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99968a1ddd7416bb0b736c6b914aa1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.913251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99968a1ddd7416bb0b736c6b914aa1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.916055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99968a1ddd7416bb0b736c6b914aa1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.929495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99968a1ddd7416bb0b736c6b914aa1e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:54.955045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 871/1749] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:54 2022] DELETE /identity/v3/users/e99968a1ddd7416bb0b736c6b914aa1e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:54.959413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:54.964338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:54.965102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:54.998566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:54.999951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:55.087072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:55.146167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 879/1750] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:58:55.158841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:55.160961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:55.284321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b0e48-b92f-4298-9a1a-222cc9707924 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:55.296138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:55.296797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:55.297079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:55.297354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:55.318818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:55.320310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:55.346193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 872/1751] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:56.215485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:56.217471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:56.249283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f91351-65da-4114-9e1f-a1d2eb3ec5e1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:56.263882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:56.265214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:56.265644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:56.266004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] PATH_INFO: `/v3/projects/aba2cec813434bd2a1e3adb2fc6d7bc7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:56.267984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba2cec813434bd2a1e3adb2fc6d7bc7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:56.274107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:56.405057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba2cec813434bd2a1e3adb2fc6d7bc7', 'request_id': 'req-d12377db-c57c-438c-a98e-7497691a0031'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.423660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba2cec813434bd2a1e3adb2fc6d7bc7', 'request_id': 'req-d12377db-c57c-438c-a98e-7497691a0031'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.448467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba2cec813434bd2a1e3adb2fc6d7bc7', 'request_id': 'req-d12377db-c57c-438c-a98e-7497691a0031'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.476349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] The token cache is being invalidate because project aba2cec813434bd2a1e3adb2fc6d7bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:56.477785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.481196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d12377db-c57c-438c-a98e-7497691a0031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.487178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 880/1752] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:56 2022] DELETE /identity/v3/projects/aba2cec813434bd2a1e3adb2fc6d7bc7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:56.487770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:56.512486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e50a9-304e-4ac0-b153-6379d0b56cbc service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:56.690429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:56.692511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:56.692932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:56.694085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] PATH_INFO: `/v3/users/de43eb03946a4b4b9585e79860424b65` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:56.697243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=de43eb03946a4b4b9585e79860424b65)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:56.702943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:56.785743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:56.821912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de43eb03946a4b4b9585e79860424b65'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.833385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de43eb03946a4b4b9585e79860424b65'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.871471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de43eb03946a4b4b9585e79860424b65'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.874571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de43eb03946a4b4b9585e79860424b65'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.886044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de43eb03946a4b4b9585e79860424b65'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:56.906755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 873/1753] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:56 2022] DELETE /identity/v3/users/de43eb03946a4b4b9585e79860424b65 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:58:57.439100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:57.439746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:57.439746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:57.505131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:57.506498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:57.656868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 881/1754] 10.222.0.27 () {58 vars in 1015 bytes} [Thu Sep 15 12:58:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:58:57.819666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:57.829747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:57.880799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b15b93-0a58-49dc-829f-4a21dba3b06a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:57.904761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:57.907206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:57.907493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:57.907766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:57.944979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:57.958745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:58.008437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 874/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:58:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:58:59.227009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:59.229256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:59.493502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:58:59.497525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67017efc-05cc-4c79-9c1b-35f01926d309 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:59.497675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:59.504782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:59.505453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:59.506124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:59.506399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] PATH_INFO: `/v3/projects/4f82f3da3aa941ceb6446053cf60a840` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:59.512141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f82f3da3aa941ceb6446053cf60a840)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:59.517322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:59.539908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c34b9-ace8-4a24-ae16-b4c08cfa6b78 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:58:59.553187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:58:59.554247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:59.554878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:59.555321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] PATH_INFO: `/v3/users/3a32bdcd6f154d83a508920b546a86de` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:59.559497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a32bdcd6f154d83a508920b546a86de)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:58:59.564024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:58:59.566590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:58:59.693719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f82f3da3aa941ceb6446053cf60a840', 'request_id': 'req-746e0aad-bb4a-49a5-b25b-d1a815bb5514'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.714727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f82f3da3aa941ceb6446053cf60a840', 'request_id': 'req-746e0aad-bb4a-49a5-b25b-d1a815bb5514'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.721894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a32bdcd6f154d83a508920b546a86de'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.744546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f82f3da3aa941ceb6446053cf60a840', 'request_id': 'req-746e0aad-bb4a-49a5-b25b-d1a815bb5514'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.747031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a32bdcd6f154d83a508920b546a86de'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.790732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] The token cache is being invalidate because project 4f82f3da3aa941ceb6446053cf60a840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:58:59.791647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.793062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-746e0aad-bb4a-49a5-b25b-d1a815bb5514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.796972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 882/1756] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:58:59 2022] DELETE /identity/v3/projects/4f82f3da3aa941ceb6446053cf60a840 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:58:59.797844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a32bdcd6f154d83a508920b546a86de'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.801499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a32bdcd6f154d83a508920b546a86de'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.802782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9fce-2fa6-456c-83d3-e5a01a47b4b0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:59.805281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9fce-2fa6-456c-83d3-e5a01a47b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:59.805857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f9fce-2fa6-456c-83d3-e5a01a47b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:59.827389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-37153566-3e7a-47c3-b07c-b8f01be42be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a32bdcd6f154d83a508920b546a86de'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:58:59.847697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 875/1757] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:58:59 2022] DELETE /identity/v3/users/3a32bdcd6f154d83a508920b546a86de => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:58:59.856170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:58:59.856557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:58:59.856848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:58:59.872964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-576f9fce-2fa6-456c-83d3-e5a01a47b4b0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:59.874188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-576f9fce-2fa6-456c-83d3-e5a01a47b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:58:59.941220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:58:59.948221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:00.079058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 883/1758] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:00.092700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:00.093141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:00.093526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:00.175705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:00.181079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:00.184866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 876/1759] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:00.214127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 884/1760] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:00.214978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:00.269007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9da36-9308-47dc-a2a8-8a8e7251cc86 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:00.293648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:00.297240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:00.298257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:00.298635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:00.299619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:00.302682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:00.309076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:00.315817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:00.354516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:00.356933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:00.389123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 877/1761] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:00.400993 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:00.455826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f674b222-a78e-4888-aefd-e43ace7f59d7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:00.490093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:00.494922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:00.494922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:00.495658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:00.499134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:00.507738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:00.581221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f83430-a94a-4f48-90c5-70f3628cdd01 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:00.606982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:00.625899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:00.626268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:00.626558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:00.645516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 878/1762] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:00 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:00.655745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:00.703179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:00.713154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:00.798806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 885/1763] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:00.966539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e8534-c9e4-4095-8bd8-1078d58cb28b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:00.987006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:00.987732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:00.988086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:00.988457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:00.989173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:00.991278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:01.066578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 879/1764] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:00 2022] POST /identity/v3/users => generated 291 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:01.069205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:01.114602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9d5c0-72a1-411a-8995-1544b7495b53 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:01.142900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:01.146557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:01.146557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:01.146557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:01.146557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:01.147793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:01.174125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 886/1765] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:01.175148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:01.192301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfcbb3-aba2-4740-adf1-e8b69d78cc60 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:01.209053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b431942-4ac1-40b2-863a-30e66ded3f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:01.210297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431942-4ac1-40b2-863a-30e66ded3f00 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:01.210297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431942-4ac1-40b2-863a-30e66ded3f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:01.210297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b431942-4ac1-40b2-863a-30e66ded3f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:01.210954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b431942-4ac1-40b2-863a-30e66ded3f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:01.213396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b431942-4ac1-40b2-863a-30e66ded3f00 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:01.257478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:01.264555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 880/1766] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:01.274086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e47163-335d-4b6b-87e3-9d2b22721963 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:01.302454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:01.304102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:01.304603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:01.305040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] PATH_INFO: `/v3/projects/99b144f7629744b4b831af88419caa52/users/f1c2243b9c8e42cfa7202a9e5d9016c5/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:01.348910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b144f7629744b4b831af88419caa52, user_id=f1c2243b9c8e42cfa7202a9e5d9016c5, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:01.353639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:01.378652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:01.388757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 887/1767] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:01 2022] PUT /identity/v3/projects/99b144f7629744b4b831af88419caa52/users/f1c2243b9c8e42cfa7202a9e5d9016c5/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:01.401363 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:01.401774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:01.402147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:01.472884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] MFA Rules not processed for user `f1c2243b9c8e42cfa7202a9e5d9016c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:01.475107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:01.669683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 881/1768] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:01.671480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:01.846203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413abf9f-2f27-4954-8f71-b946e0740c40 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:01.874598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:01.875270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:01.875554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:01.875879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:01.884885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:01.902308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:02.011202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 888/1769] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:02.017909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:02.039878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd3a15-c5a1-43f0-8c2c-67e020cf9fc6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:02.263110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:02.349333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:02.350395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:02.351491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:02.351817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:02.353769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:02.365825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:02.431747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 882/1770] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:02 2022] POST /identity/v3/users => generated 297 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:02.432642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:02.449847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c808e06-4438-4160-b0a2-c7cef8ba0ada admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:02.456908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:02.457778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:02.458077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:02.458352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:02.459010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:02.466065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:02.522402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:02.529846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 889/1771] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:02.537595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844d577-2250-4891-8fb5-bc646d82fda8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:02.551537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:02.554487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:02.554789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:02.556306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] PATH_INFO: `/v3/projects/261da5a5aa7d4fa59367c3ae13917cf5/users/7f1ada52ac054262a3c1c35eff5ef0b7/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:02.583787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:02.589141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:02.659982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261da5a5aa7d4fa59367c3ae13917cf5, user_id=7f1ada52ac054262a3c1c35eff5ef0b7, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:02.661122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c2adc-8713-4194-ab60-fbfc40925693 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:02.678032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:02.688637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:02.689300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:02.689582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:02.689859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] PATH_INFO: `/v3/users/adb0bead59604e0592c7e12ec3e9409a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:02.694671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb0bead59604e0592c7e12ec3e9409a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:02.699861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:59:02.699861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 15 12:59:02.699861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: Sep 15 12:59:02.712698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:02.724523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 883/1772] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:02 2022] PUT /identity/v3/projects/261da5a5aa7d4fa59367c3ae13917cf5/users/7f1ada52ac054262a3c1c35eff5ef0b7/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:02.726027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:02.924663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb0bead59604e0592c7e12ec3e9409a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:02.984899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d489f3-ee51-41c8-bca5-9032106ffc12 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:02.985764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb0bead59604e0592c7e12ec3e9409a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:02.998061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:02.998783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:02.999282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:02.999709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:03.001101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:03.004453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:03.020839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb0bead59604e0592c7e12ec3e9409a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:03.033873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 884/1773] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:03.037797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:03.045603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb0bead59604e0592c7e12ec3e9409a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:03.046998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02e31893-a419-4301-8e95-50917a8a4c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb0bead59604e0592c7e12ec3e9409a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:03.061461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5072834-e06c-4cd7-a8c9-c96baa291376 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:03.068262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 890/1774] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:02 2022] DELETE /identity/v3/users/adb0bead59604e0592c7e12ec3e9409a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:03.070256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441f44-270c-4971-9b17-80eca42b2845 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:03.070727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441f44-270c-4971-9b17-80eca42b2845 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:03.071214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e441f44-270c-4971-9b17-80eca42b2845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:03.097086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9e441f44-270c-4971-9b17-80eca42b2845 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:03.098468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9e441f44-270c-4971-9b17-80eca42b2845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:03.174194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:03.213663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 891/1775] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:03.330075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20653ef-ced0-4d92-9c23-e405ca666c10 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:03.330501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20653ef-ced0-4d92-9c23-e405ca666c10 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:03.330918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20653ef-ced0-4d92-9c23-e405ca666c10 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:03.333270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 892/1776] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:59:03.355474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9b1c1-9eb8-4bbf-aab3-2c647e860025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:03.356066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9b1c1-9eb8-4bbf-aab3-2c647e860025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:03.359275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9b1c1-9eb8-4bbf-aab3-2c647e860025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:03.359548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9b1c1-9eb8-4bbf-aab3-2c647e860025 admin admin] PATH_INFO: `/v3/projects/261da5a5aa7d4fa59367c3ae13917cf5/users/7f1ada52ac054262a3c1c35eff5ef0b7/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:03.360042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b892-8c30-47c1-887f-522610636d4d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:03.360359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b892-8c30-47c1-887f-522610636d4d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:03.360625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737b892-8c30-47c1-887f-522610636d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:03.363422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9b1c1-9eb8-4bbf-aab3-2c647e860025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=261da5a5aa7d4fa59367c3ae13917cf5, user_id=7f1ada52ac054262a3c1c35eff5ef0b7, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:03.376634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9b1c1-9eb8-4bbf-aab3-2c647e860025 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:03.397070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 885/1777] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:03 2022] PUT /identity/v3/projects/261da5a5aa7d4fa59367c3ae13917cf5/users/7f1ada52ac054262a3c1c35eff5ef0b7/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:03.409005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:03.409306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:03.409625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:03.439684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e737b892-8c30-47c1-887f-522610636d4d None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:03.443478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e737b892-8c30-47c1-887f-522610636d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:03.491471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] MFA Rules not processed for user `7f1ada52ac054262a3c1c35eff5ef0b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:03.503692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:03.630758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:03.754811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 893/1778] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:03.757763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737b892-8c30-47c1-887f-522610636d4d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:03.762326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e737b892-8c30-47c1-887f-522610636d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:03.823224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 886/1779] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:03.834241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:03.839682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:03.973361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737b892-8c30-47c1-887f-522610636d4d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:03.980913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:03.981800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:03.982189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:03.982727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.005301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce7574-3f16-46f8-ad0b-51528e089deb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:04.017057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:04.017714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:04.017984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:04.018260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.047063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:04.048688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:04.077646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:04.078587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 887/1780] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:04.079566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:04.085451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:04.086984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:04.101592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 894/1781] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:04.160687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4c83e-3e62-47e6-8ce0-4bf8722e69a5 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:04.195008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:04.195124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:04.200526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:04.200882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] PATH_INFO: `/v3/projects/f228dcd0990f4d33a8c145933b572005` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.202273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f228dcd0990f4d33a8c145933b572005)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:04.208347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:04.245648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:04.247786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:04.292431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3ffab-027d-44d3-8612-a683848da27f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:04.308479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5223d-8c36-448c-b64c-7314c0deba39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:04.309120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5223d-8c36-448c-b64c-7314c0deba39 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:04.309398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5223d-8c36-448c-b64c-7314c0deba39 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:04.309675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5223d-8c36-448c-b64c-7314c0deba39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.343607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5223d-8c36-448c-b64c-7314c0deba39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:04.347508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5223d-8c36-448c-b64c-7314c0deba39 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:04.356017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f228dcd0990f4d33a8c145933b572005', 'request_id': 'req-fdddacce-4619-486b-bbbe-2e537b00dfdc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:04.372886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f228dcd0990f4d33a8c145933b572005', 'request_id': 'req-fdddacce-4619-486b-bbbe-2e537b00dfdc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:04.378633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 895/1782] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:04.389239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f228dcd0990f4d33a8c145933b572005', 'request_id': 'req-fdddacce-4619-486b-bbbe-2e537b00dfdc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:04.437088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] The token cache is being invalidate because project f228dcd0990f4d33a8c145933b572005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:04.437088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:04.437088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdddacce-4619-486b-bbbe-2e537b00dfdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:04.437088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 888/1783] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:04 2022] DELETE /identity/v3/projects/f228dcd0990f4d33a8c145933b572005 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:04.444473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:04.444473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:04.444473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.492191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:04.496611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:04.544585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:04.716925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 896/1784] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:04.725217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:04.725789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:04.726227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.841788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:04.843057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:04.885651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 889/1785] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:04.889692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:04.904659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c89368-644c-49a8-a203-cb81408c3b46 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:04.920063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:04.921873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:04.922047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:04.935023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:04.935842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:04.936161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:04.938446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:04.971769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 897/1786] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:04.976155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:04.992123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeed831-0a31-48eb-a3e1-2f9fb261d3b2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.012593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.013874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.013874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.014120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.014788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:05.017781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:05.087484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 890/1787] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:04 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:05.090268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.103098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fc469-3392-41dc-8b75-186e40ba5cd1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.250963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.251863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.252225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.252737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.253851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:05.256875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:05.272244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.278026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:05.308588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b8c29-f2c2-4714-90c9-983ed285d89c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.314939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 898/1788] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:05 2022] POST /identity/v3/users => generated 303 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:05.316699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.324681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.333616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30484af-98cc-4821-8554-30a5a0d15cef admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.339649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.340302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.340620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.340893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.340990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.341195 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.341817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:05.342143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] PATH_INFO: `/v3/users/f1c2243b9c8e42cfa7202a9e5d9016c5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.344637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c2243b9c8e42cfa7202a9e5d9016c5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:05.344815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:05.346718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:05.368182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:05.379111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 899/1789] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:05.380635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.398715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4df64a-08c4-4d83-902e-6c323d77df69 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.407008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.407454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.407744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.408118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.409357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:05.411777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:05.442101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 900/1790] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:05.443653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.469570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c2243b9c8e42cfa7202a9e5d9016c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:05.488170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c2243b9c8e42cfa7202a9e5d9016c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:05.489064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9bcf5-e762-4835-ba75-b47b721a9c12 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.498090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a585f94d-ef8a-4fce-8a6f-2d14c7450f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.499167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f94d-ef8a-4fce-8a6f-2d14c7450f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.500041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f94d-ef8a-4fce-8a6f-2d14c7450f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.500559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585f94d-ef8a-4fce-8a6f-2d14c7450f20 admin admin] PATH_INFO: `/v3/projects/3a962a4a42c1442ab39dc5280d2af86a/users/4ed058f5f2fd45ed96176d4910d58501/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.522835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585f94d-ef8a-4fce-8a6f-2d14c7450f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a962a4a42c1442ab39dc5280d2af86a, user_id=4ed058f5f2fd45ed96176d4910d58501, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:05.526068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c2243b9c8e42cfa7202a9e5d9016c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:05.526925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c2243b9c8e42cfa7202a9e5d9016c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:05.528166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a585f94d-ef8a-4fce-8a6f-2d14c7450f20 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:05.537827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4850606e-c6fd-425f-ae02-a452b9431e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c2243b9c8e42cfa7202a9e5d9016c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:05.551337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 901/1791] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:05 2022] PUT /identity/v3/projects/3a962a4a42c1442ab39dc5280d2af86a/users/4ed058f5f2fd45ed96176d4910d58501/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:05.555915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.557055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.557664 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.558123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 891/1792] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:05 2022] DELETE /identity/v3/users/f1c2243b9c8e42cfa7202a9e5d9016c5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:05.562275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.562859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.563408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:05.584190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:05.585876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:05.598628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:05.600130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:05.650781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:05.670160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 902/1793] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:05.770616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.772302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:05.797044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 892/1794] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:05.847620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:05.849705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:05.952077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7322132-221d-4f08-afbb-7a87f286caa9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:05.970046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2c7ff-4957-49cb-955a-2b2e575cf21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:05.970716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2c7ff-4957-49cb-955a-2b2e575cf21a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:05.970961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2c7ff-4957-49cb-955a-2b2e575cf21a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:05.971271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2c7ff-4957-49cb-955a-2b2e575cf21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:06.004806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2c7ff-4957-49cb-955a-2b2e575cf21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:06.006436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2c7ff-4957-49cb-955a-2b2e575cf21a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:06.039623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9e546-d72a-441f-8663-d1a403e0658d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:06.041526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 903/1795] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:06.046247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c882c-9458-41ce-8bde-b227b633e5df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:06.047104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c882c-9458-41ce-8bde-b227b633e5df service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:06.047494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c882c-9458-41ce-8bde-b227b633e5df service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:06.047897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c882c-9458-41ce-8bde-b227b633e5df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:06.060134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c882c-9458-41ce-8bde-b227b633e5df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:06.062051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c882c-9458-41ce-8bde-b227b633e5df service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:06.079771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 893/1796] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:06.374264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeda254-112d-4fd8-8784-a78c88d091e3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:06.375102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeda254-112d-4fd8-8784-a78c88d091e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:06.375772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeda254-112d-4fd8-8784-a78c88d091e3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:06.379254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 904/1797] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:59:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:59:06.428427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:06.434252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:06.434816 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:06.492201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:06.498651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:06.593944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:06.690349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 894/1798] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:59:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:06.865363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeda254-112d-4fd8-8784-a78c88d091e3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:06.873205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aaeda254-112d-4fd8-8784-a78c88d091e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:06.952910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeda254-112d-4fd8-8784-a78c88d091e3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:06.985276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:06.986235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:06.986392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:06.986741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] PATH_INFO: `/v3/projects/61a38a268b9547ad86f9a7bfca700fb3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:06.988432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a38a268b9547ad86f9a7bfca700fb3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:06.992183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:07.152244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:07.153687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:07.176475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea576c-4d87-4354-a901-ef362e4af27c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:07.205253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:07.206024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:07.206556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:07.207043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:07.229378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:07.231361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:07.252838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 895/1799] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:07.263883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a38a268b9547ad86f9a7bfca700fb3', 'request_id': 'req-ceb99014-9061-4daf-8d97-1425f862810a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.289115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a38a268b9547ad86f9a7bfca700fb3', 'request_id': 'req-ceb99014-9061-4daf-8d97-1425f862810a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.327125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a38a268b9547ad86f9a7bfca700fb3', 'request_id': 'req-ceb99014-9061-4daf-8d97-1425f862810a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.368292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] The token cache is being invalidate because project 61a38a268b9547ad86f9a7bfca700fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:07.371533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.375863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.381627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 905/1800] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:06 2022] DELETE /identity/v3/projects/61a38a268b9547ad86f9a7bfca700fb3 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:07.386222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:07.406501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8db455-d806-44d4-8ef4-584a2b3438b3 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:07.590331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:07.592010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:07.592612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:07.594297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] PATH_INFO: `/v3/users/1824f20481264f32b81b9008d6de134e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:07.598131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1824f20481264f32b81b9008d6de134e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:07.606417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:07.700307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1824f20481264f32b81b9008d6de134e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.715393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1824f20481264f32b81b9008d6de134e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.735095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1824f20481264f32b81b9008d6de134e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.736162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1824f20481264f32b81b9008d6de134e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.750144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1824f20481264f32b81b9008d6de134e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:07.764305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 896/1801] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:07 2022] DELETE /identity/v3/users/1824f20481264f32b81b9008d6de134e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:08.899155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:08.902312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:08.918570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb99014-9061-4daf-8d97-1425f862810a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:09.011143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:09.013111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.015506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.015506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] PATH_INFO: `/v3/projects/99b144f7629744b4b831af88419caa52` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.019534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b144f7629744b4b831af88419caa52)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:09.035635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:09.062796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:09.142226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b144f7629744b4b831af88419caa52', 'request_id': 'req-59188efc-2770-4fac-a01a-ef110babc05e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.152614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b144f7629744b4b831af88419caa52', 'request_id': 'req-59188efc-2770-4fac-a01a-ef110babc05e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.159780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b144f7629744b4b831af88419caa52', 'request_id': 'req-59188efc-2770-4fac-a01a-ef110babc05e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.185830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] The token cache is being invalidate because project 99b144f7629744b4b831af88419caa52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:09.186305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.187161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.192298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 906/1802] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:08 2022] DELETE /identity/v3/projects/99b144f7629744b4b831af88419caa52 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:09.193303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:09.200744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:09.211341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:09.214929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:09.220884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca57fe-634a-4511-aa57-5ab00c70d5a9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:09.281347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:09.331370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:09.332939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.333336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.334018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] PATH_INFO: `/v3/users/7f1ada52ac054262a3c1c35eff5ef0b7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.336221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1ada52ac054262a3c1c35eff5ef0b7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:09.340134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59188efc-2770-4fac-a01a-ef110babc05e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:09.341484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:09.350989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:09.351825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.352713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.353073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] PATH_INFO: `/v3/projects/9f29ae38596f499d83c4f2a5c503440a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.354496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f29ae38596f499d83c4f2a5c503440a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:09.356724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:09.463666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1ada52ac054262a3c1c35eff5ef0b7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.477092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1ada52ac054262a3c1c35eff5ef0b7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.479235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f29ae38596f499d83c4f2a5c503440a', 'request_id': 'req-70313f56-256b-4402-b55d-a7a7b091fd7c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.486475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f29ae38596f499d83c4f2a5c503440a', 'request_id': 'req-70313f56-256b-4402-b55d-a7a7b091fd7c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.496001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f29ae38596f499d83c4f2a5c503440a', 'request_id': 'req-70313f56-256b-4402-b55d-a7a7b091fd7c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.505511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1ada52ac054262a3c1c35eff5ef0b7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.506264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1ada52ac054262a3c1c35eff5ef0b7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.517395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] The token cache is being invalidate because project 9f29ae38596f499d83c4f2a5c503440a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:09.518345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.519071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-70313f56-256b-4402-b55d-a7a7b091fd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.521809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0671604c-c33a-4446-964e-7f010381239e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1ada52ac054262a3c1c35eff5ef0b7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:09.522253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 907/1803] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:09 2022] DELETE /identity/v3/projects/9f29ae38596f499d83c4f2a5c503440a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:09.528883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.528883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.528883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.535376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 897/1804] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:09 2022] DELETE /identity/v3/users/7f1ada52ac054262a3c1c35eff5ef0b7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:09.582927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:09.584368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:09.633166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:09.701714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 908/1805] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:09.704588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.704879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.705181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.727196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:09.729053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:09.744991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 898/1806] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:09.746218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:09.771179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f28f24-36a8-460f-9e4e-30ebb2716b4e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:09.806415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:09.807483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.808618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.809728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.811941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:09.813183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:09.817015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:09.845152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 909/1807] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:09.851524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:09.865585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d8829-2c84-49ca-8854-94b1341182ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:09.877701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5e78a-16d0-4309-8b9e-99231e54a58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:09.881141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5e78a-16d0-4309-8b9e-99231e54a58f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:09.881456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5e78a-16d0-4309-8b9e-99231e54a58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:09.883425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5e78a-16d0-4309-8b9e-99231e54a58f admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:09.884886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5e78a-16d0-4309-8b9e-99231e54a58f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:09.887418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5e78a-16d0-4309-8b9e-99231e54a58f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:09.983682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 899/1808] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:09 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:09.988817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:09.999954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.000677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.001302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.009647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ba00b-d732-46bc-a710-ca2bb4b7f484 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.066564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] MFA Rules not processed for user `4ed058f5f2fd45ed96176d4910d58501`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:10.073567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:10.205944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.206578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.207550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.207981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.208814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.210949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.254119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 910/1809] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:09 2022] POST /identity/v3/users => generated 282 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:10.258007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.274017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ec532-5961-45b0-826f-be8496d9bb24 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.281493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 900/1810] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:10.282755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.284532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:10.288164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.288907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.289295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.291619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.292434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.298717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.303158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89312b12-28ac-49d5-b04a-876ac3453048 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.309036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.309802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.310731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.311093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.311814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.314212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.328150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 911/1811] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:10.331325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:10.331705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.356321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784221f-d713-4fb4-a418-801587ae7cc5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.364264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 901/1812] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:10 2022] POST /identity/v3/projects => generated 405 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:10.365273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.440087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462055e2-6056-4ae9-a24a-4bebbbf6f1c0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.445505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.446422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.447221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.447567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.449246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.449406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.449499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.449499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.449499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.450623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.450934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.452871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.467470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 912/1813] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:10.473209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.479894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 902/1814] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:10 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:10.483989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.489821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc059ee7-180e-49e3-9c4a-64f8cafc0baa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.511313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb5bb4-5086-4a73-8ca2-17996bc8c88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.512179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb5bb4-5086-4a73-8ca2-17996bc8c88d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.512596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb5bb4-5086-4a73-8ca2-17996bc8c88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.514255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb5bb4-5086-4a73-8ca2-17996bc8c88d admin admin] PATH_INFO: `/v3/projects/0a6a2e9ab2174c90a93425ddb8cf2c61/users/7ee3b845fcbd4df2a60c02ebc3063561/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.517510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ddad-99a1-40a2-a2db-b3b993e26e60 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.525515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.526274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.526711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.527470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.528283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.530689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.538033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb5bb4-5086-4a73-8ca2-17996bc8c88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6a2e9ab2174c90a93425ddb8cf2c61, user_id=7ee3b845fcbd4df2a60c02ebc3063561, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.542930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb5bb4-5086-4a73-8ca2-17996bc8c88d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.547062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 903/1815] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:10.548635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.577071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cfc2c-7e50-47a7-99da-604a7fbbcf52 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.583226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 913/1816] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:10 2022] PUT /identity/v3/projects/0a6a2e9ab2174c90a93425ddb8cf2c61/users/7ee3b845fcbd4df2a60c02ebc3063561/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:10.594085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.594378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.594656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.624031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:10.640314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] MFA Rules not processed for user `7ee3b845fcbd4df2a60c02ebc3063561`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:10.642061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:10.682326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.683252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.683674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.684098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] PATH_INFO: `/v3/projects/c2ca73dc54964f3fac41545b0bf21b61/users/ffe8c2fe6625431f9f1073df35eb127b/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.705344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ca73dc54964f3fac41545b0bf21b61, user_id=ffe8c2fe6625431f9f1073df35eb127b, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.709447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.733511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 904/1817] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:10 2022] PUT /identity/v3/projects/c2ca73dc54964f3fac41545b0bf21b61/users/ffe8c2fe6625431f9f1073df35eb127b/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:10.737405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.749398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 914/1818] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:10.756122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.776548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbca20-b6ec-44aa-8492-53f9d26858f4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.870437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.872088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.872414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.872698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.873362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.880530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.907332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:10.923676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9e17-ef6f-465d-83c9-77b5c5484674 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:10.930181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:10.932933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:10.934770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:10.935475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:10.936401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:10.938544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:10.944418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 915/1819] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:10 2022] POST /identity/v3/projects => generated 363 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:10.949095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.955620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 905/1820] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:10.960223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:10.982990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229716-aa6c-4d08-a688-efb02801ccd1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.109126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6754a50-7459-4fd4-93c1-703017f2d8d4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.115776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.116537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.116826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.117259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.118198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.119037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.120241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.120388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.120955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.121309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] PATH_INFO: `/v3/projects/c2ca73dc54964f3fac41545b0bf21b61/users/ffe8c2fe6625431f9f1073df35eb127b/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.124097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ca73dc54964f3fac41545b0bf21b61, user_id=ffe8c2fe6625431f9f1073df35eb127b, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.128087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.143254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 906/1821] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:10 2022] PUT /identity/v3/projects/c2ca73dc54964f3fac41545b0bf21b61/users/ffe8c2fe6625431f9f1073df35eb127b/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:11.156301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 916/1822] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:10 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:11.161017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.241073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4888bb-6041-4a0d-92fe-788320f912b3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.247135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.247777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.248242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.248715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.249377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.251005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.263946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.265862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:11.287373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0758d14-133a-4d4a-bcbc-21c49569facd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.288105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 907/1823] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:11.289661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.292647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.293795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.294092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.294424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] PATH_INFO: `/v3/projects/261da5a5aa7d4fa59367c3ae13917cf5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.295778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] RBAC: Authorizing `identity:delete_project(project_id=261da5a5aa7d4fa59367c3ae13917cf5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.297830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.312053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:11.314949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47def26d-0305-4009-a903-1c04e77b896b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.329408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.330212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.330559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.330902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] PATH_INFO: `/v3/projects/3cd7e35b0fd8468c8b6128a90bdc8bc5/users/105795a0f85040c8b12f477967328d72/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.367509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261da5a5aa7d4fa59367c3ae13917cf5', 'request_id': 'req-6f3135a9-44f4-468e-8051-44fcf114451a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:11.382865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cd7e35b0fd8468c8b6128a90bdc8bc5, user_id=105795a0f85040c8b12f477967328d72, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.386830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.409744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261da5a5aa7d4fa59367c3ae13917cf5', 'request_id': 'req-6f3135a9-44f4-468e-8051-44fcf114451a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:11.415594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 908/1824] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:11 2022] PUT /identity/v3/projects/3cd7e35b0fd8468c8b6128a90bdc8bc5/users/105795a0f85040c8b12f477967328d72/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:11.421091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261da5a5aa7d4fa59367c3ae13917cf5', 'request_id': 'req-6f3135a9-44f4-468e-8051-44fcf114451a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:11.426415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.450453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] The token cache is being invalidate because project 261da5a5aa7d4fa59367c3ae13917cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:11.450765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:11.451581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6f3135a9-44f4-468e-8051-44fcf114451a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:11.458057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 917/1825] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:11 2022] DELETE /identity/v3/projects/261da5a5aa7d4fa59367c3ae13917cf5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:11.464699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfbb88-d40b-4f5d-b698-50ef859186a5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.465295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfbb88-d40b-4f5d-b698-50ef859186a5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.466644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfbb88-d40b-4f5d-b698-50ef859186a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.503780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:11.505351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-09bfbb88-d40b-4f5d-b698-50ef859186a5 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:11.513368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09bfbb88-d40b-4f5d-b698-50ef859186a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:11.526925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 918/1826] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:11.530097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a63e5-b0ac-45fd-9564-237874963b29 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.534725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.536148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.536148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.541996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.543854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.544470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.544968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.546379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.549198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.565992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 909/1827] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:11.570415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.572256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:11.577607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:11.579878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:11.590658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968a8f0-b578-4d4e-992f-2742b95a5a5f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.597738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b52105-cc1b-43e9-b3f2-b39e14615e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.599221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b52105-cc1b-43e9-b3f2-b39e14615e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.599221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b52105-cc1b-43e9-b3f2-b39e14615e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.599393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b52105-cc1b-43e9-b3f2-b39e14615e86 admin admin] PATH_INFO: `/v3/projects/3cd7e35b0fd8468c8b6128a90bdc8bc5/users/105795a0f85040c8b12f477967328d72/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.604020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b52105-cc1b-43e9-b3f2-b39e14615e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cd7e35b0fd8468c8b6128a90bdc8bc5, user_id=105795a0f85040c8b12f477967328d72, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.606223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 919/1828] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:11.607705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b52105-cc1b-43e9-b3f2-b39e14615e86 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.610559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.634581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 910/1829] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:11 2022] PUT /identity/v3/projects/3cd7e35b0fd8468c8b6128a90bdc8bc5/users/105795a0f85040c8b12f477967328d72/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:11.637282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce1ced-b7ab-436d-af84-c48d81623db9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.638107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.638986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.640125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.716918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] MFA Rules not processed for user `105795a0f85040c8b12f477967328d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:11.718515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:11.745925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.746495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.746840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.747601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.748554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:11.748861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.750940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.771678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 920/1830] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:59:11.779290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.810694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e3c12-4f79-4ca6-b1e2-0f8261327ffd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:11.827230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:11.828025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:11.828159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:11.828478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:11.829139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:11.831650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:11.851569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 911/1831] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:11.869965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.872331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:11.893676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 921/1832] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:11 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:11.898063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:11.978169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd04e0-fee6-4d04-8de3-5acff1106c99 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.047265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:12.054017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10321358-528e-4dc8-82cb-c7738d9b5478 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.055902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.056840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.057154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.057433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.063436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.064236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.064643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.065131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.065740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.067988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.106717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:12.116394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 922/1833] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:11 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:12.122340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.138350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae664fb3-4ca9-4d4c-b6d4-1d9c6b00c50d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.148754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.150483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.151950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.152309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.152919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.154581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.165019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.166900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.175409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 923/1834] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:12.177862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 912/1835] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:12.179317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.190376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316ecae-2cff-47c8-bfba-915bfdb3fbdb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.195456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.196066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.197058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.197379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.197998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.199653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.212843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 924/1836] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:12.218850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.232745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea3d69-9735-4baa-a045-5db7516da045 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.238438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7935c8-11aa-4a76-afb8-71f1bc034d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.239159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7935c8-11aa-4a76-afb8-71f1bc034d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.242412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7935c8-11aa-4a76-afb8-71f1bc034d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.242412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7935c8-11aa-4a76-afb8-71f1bc034d70 admin admin] PATH_INFO: `/v3/projects/7002666003b44a94a9feded3951a798b/users/8c6b9ac92099482997d62ef8c009f066/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.262660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7935c8-11aa-4a76-afb8-71f1bc034d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7002666003b44a94a9feded3951a798b, user_id=8c6b9ac92099482997d62ef8c009f066, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.265424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7935c8-11aa-4a76-afb8-71f1bc034d70 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.276957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.291079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2caa2-e869-46e8-97e9-db82dfbba7f8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.298642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 913/1837] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:12 2022] PUT /identity/v3/projects/7002666003b44a94a9feded3951a798b/users/8c6b9ac92099482997d62ef8c009f066/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:12.310674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.311114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.311492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.358186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] MFA Rules not processed for user `8c6b9ac92099482997d62ef8c009f066`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:12.360203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:12.363908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.364649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.364997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.365359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.367253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:12.452266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 914/1838] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:12.456758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.473423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.477418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.500038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 925/1839] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:12.560320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc74ec-7f76-443f-a277-0eb3d877bfb7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.566337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.566955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.567692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.567976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.568717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.572288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.609506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 915/1840] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:12 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:12.613542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.630099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fb95d-c254-4806-9e5b-78816f41e744 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.739149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87375adc-2f15-4fde-967d-09dd985378af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.739764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87375adc-2f15-4fde-967d-09dd985378af admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.740025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87375adc-2f15-4fde-967d-09dd985378af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.740331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87375adc-2f15-4fde-967d-09dd985378af admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.740963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87375adc-2f15-4fde-967d-09dd985378af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.742888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87375adc-2f15-4fde-967d-09dd985378af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.773352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 926/1841] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:12 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:12.778427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.797306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcaaba-9409-4a02-9e8e-639db85d3486 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.803482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.804158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.804478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.804748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.805448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.807126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.826547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.826848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.827115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.832327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 916/1842] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:12.837513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.854912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:12.858002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea6608-027b-4dcb-9cd7-217bf09692a5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:12.866872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] MFA Rules not processed for user `ffe8c2fe6625431f9f1073df35eb127b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:12.868529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:12.876176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:12.876883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:12.877167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:12.877447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] PATH_INFO: `/v3/projects/1ab7522cbd6a4334934ea00ca327e497/users/433f45c4db8046ab956240ed27d7a76a/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:12.881081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:12.896323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab7522cbd6a4334934ea00ca327e497, user_id=433f45c4db8046ab956240ed27d7a76a, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:12.899267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:12.923632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 917/1843] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:12 2022] PUT /identity/v3/projects/1ab7522cbd6a4334934ea00ca327e497/users/433f45c4db8046ab956240ed27d7a76a/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:12.924587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:12.998063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 927/1844] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:13.010645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce10510-31f4-4b54-a2a2-5d72ea46ad63 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:13.015492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dc8f2-1c4e-4d8a-8afe-d2458c0a1284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:13.016080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dc8f2-1c4e-4d8a-8afe-d2458c0a1284 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.016397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dc8f2-1c4e-4d8a-8afe-d2458c0a1284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.016940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dc8f2-1c4e-4d8a-8afe-d2458c0a1284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.017706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dc8f2-1c4e-4d8a-8afe-d2458c0a1284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:13.019449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3dc8f2-1c4e-4d8a-8afe-d2458c0a1284 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:13.034422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 918/1845] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:13.035257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:13.051910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5c771-10a7-4758-b0d0-7080600d7d60 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:13.064444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:13.066563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.067059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.067643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] PATH_INFO: `/v3/projects/1ab7522cbd6a4334934ea00ca327e497/users/433f45c4db8046ab956240ed27d7a76a/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.070155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab7522cbd6a4334934ea00ca327e497, user_id=433f45c4db8046ab956240ed27d7a76a, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:13.073687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:13.085891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 928/1846] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:13 2022] PUT /identity/v3/projects/1ab7522cbd6a4334934ea00ca327e497/users/433f45c4db8046ab956240ed27d7a76a/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:13.099016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.099330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.099616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.129169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] MFA Rules not processed for user `433f45c4db8046ab956240ed27d7a76a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:13.130348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:13.221370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 919/1847] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:13.235119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:13.353321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a069-b170-4da7-83be-940a9a0d34e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:13.358112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dabcb-2249-443f-b512-091b61f64327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:13.358836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dabcb-2249-443f-b512-091b61f64327 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.359601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dabcb-2249-443f-b512-091b61f64327 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.359601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dabcb-2249-443f-b512-091b61f64327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.370809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dabcb-2249-443f-b512-091b61f64327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:13.376231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dabcb-2249-443f-b512-091b61f64327 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:13.388587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 929/1848] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:13.486001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:13.499478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67824e23-b30b-4e40-99f7-7fb1ecf2bc8f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:13.504009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e2ea7-1994-4117-8600-1f5dca9ea176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:13.504661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e2ea7-1994-4117-8600-1f5dca9ea176 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.504946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e2ea7-1994-4117-8600-1f5dca9ea176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.505246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e2ea7-1994-4117-8600-1f5dca9ea176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.564205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e2ea7-1994-4117-8600-1f5dca9ea176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:13.565674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e2ea7-1994-4117-8600-1f5dca9ea176 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:13.575068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 920/1849] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:13.866881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e7296-4231-4923-805f-9f8e0c6d0726 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.869321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e7296-4231-4923-805f-9f8e0c6d0726 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.869831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e7296-4231-4923-805f-9f8e0c6d0726 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.872238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 930/1850] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:59:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:59:13.901850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903948-cac2-4ad7-8336-a08cc844e937 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:13.902136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903948-cac2-4ad7-8336-a08cc844e937 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:13.902417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2903948-cac2-4ad7-8336-a08cc844e937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:13.916877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:13.925551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e7296-4231-4923-805f-9f8e0c6d0726 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:13.933686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e2903948-cac2-4ad7-8336-a08cc844e937 None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:13.935066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2903948-cac2-4ad7-8336-a08cc844e937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:14.078963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e7296-4231-4923-805f-9f8e0c6d0726 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:14.080678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 921/1851] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:59:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:14.083925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:14.084610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.084891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.085168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] PATH_INFO: `/v3/users/8c6b9ac92099482997d62ef8c009f066` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.086316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6b9ac92099482997d62ef8c009f066)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:14.089612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:14.095991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337db3f-b9d5-4825-90e6-0eaffdea834d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.096321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337db3f-b9d5-4825-90e6-0eaffdea834d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.096603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337db3f-b9d5-4825-90e6-0eaffdea834d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.097942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 922/1852] 10.222.0.57 () {54 vars in 961 bytes} [Thu Sep 15 12:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:59:14.111764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.111764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.111764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.138262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:14.169330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:14.171422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:14.183063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6b9ac92099482997d62ef8c009f066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:14.192889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 923/1853] 10.222.0.57 () {58 vars in 1058 bytes} [Thu Sep 15 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:14.201314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:14.214210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b2642-53ff-4dc0-9a7d-b7e63ca58dda None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:14.221200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6b9ac92099482997d62ef8c009f066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:14.225796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:14.226917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.229198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.229706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.230653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6b9ac92099482997d62ef8c009f066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:14.240886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6b9ac92099482997d62ef8c009f066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:14.242557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e3787ecf-e646-4810-8533-66fc98abd875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6b9ac92099482997d62ef8c009f066'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:14.247218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:14.249159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:14.252945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 931/1854] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:13 2022] DELETE /identity/v3/users/8c6b9ac92099482997d62ef8c009f066 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:14.254383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385953f-a6cb-4576-a30f-eed294b90fa2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.254771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385953f-a6cb-4576-a30f-eed294b90fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.255087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385953f-a6cb-4576-a30f-eed294b90fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.263647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 924/1855] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:14.278657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8385953f-a6cb-4576-a30f-eed294b90fa2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:14.280000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8385953f-a6cb-4576-a30f-eed294b90fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:14.395677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 932/1856] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:14.410501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:14.412177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:14.497156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962df8b-1e5c-4f87-831e-2e11fffe9a63 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:14.555465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c357dfc-1a4e-4738-85e5-1a247e08470c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:14.556085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c357dfc-1a4e-4738-85e5-1a247e08470c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.556400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c357dfc-1a4e-4738-85e5-1a247e08470c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.556671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c357dfc-1a4e-4738-85e5-1a247e08470c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.567034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c357dfc-1a4e-4738-85e5-1a247e08470c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:14.568964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c357dfc-1a4e-4738-85e5-1a247e08470c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:14.579659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 925/1857] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:14.842405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1c7eb-d13a-4f06-80b8-e6ace93d0847 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.842775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1c7eb-d13a-4f06-80b8-e6ace93d0847 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.843126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1c7eb-d13a-4f06-80b8-e6ace93d0847 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.844571 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 933/1858] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:59:14.883373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.884349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.884628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.905777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] MFA Rules not processed for user `25816463963b44a8865c0d3693cfef88`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:14.906995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:14.950471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1c7eb-d13a-4f06-80b8-e6ace93d0847 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:14.956546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cea1c7eb-d13a-4f06-80b8-e6ace93d0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:14.964526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:14.979700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1c7eb-d13a-4f06-80b8-e6ace93d0847 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:14.990507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:14.991700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:14.991865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:14.992658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] PATH_INFO: `/v3/users/4ed058f5f2fd45ed96176d4910d58501` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:14.994788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed058f5f2fd45ed96176d4910d58501)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:14.997852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:15.008251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 926/1859] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:59:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:15.026730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:15.028458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:15.049964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50777fe5-422a-455f-bc63-61a112438a6c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:15.059295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:15.060551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:15.061223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:15.061703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:15.077386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:15.079634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:15.083973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed058f5f2fd45ed96176d4910d58501'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.113144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed058f5f2fd45ed96176d4910d58501'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.121696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed058f5f2fd45ed96176d4910d58501'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.128812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 927/1860] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:15.137639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed058f5f2fd45ed96176d4910d58501'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.140315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-feb3cd2e-546f-4414-bbad-6c632fe1e0af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed058f5f2fd45ed96176d4910d58501'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.151906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 934/1861] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:14 2022] DELETE /identity/v3/users/4ed058f5f2fd45ed96176d4910d58501 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:15.591280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:15.592862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:15.609805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45abc48-399a-4ad2-8f8d-a6399b4e6843 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:15.676860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:15.677279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:15.677655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:15.678001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] PATH_INFO: `/v3/users/7ee3b845fcbd4df2a60c02ebc3063561` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:15.679293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee3b845fcbd4df2a60c02ebc3063561)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:15.681988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:15.745931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee3b845fcbd4df2a60c02ebc3063561'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.755692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee3b845fcbd4df2a60c02ebc3063561'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.770577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee3b845fcbd4df2a60c02ebc3063561'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.771411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee3b845fcbd4df2a60c02ebc3063561'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.781918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee3b845fcbd4df2a60c02ebc3063561'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:15.799197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:15.799744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:15.799744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:15.802870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 928/1862] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:15 2022] DELETE /identity/v3/users/7ee3b845fcbd4df2a60c02ebc3063561 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:15.823525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:15.824956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:15.837451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:15.907557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 935/1863] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:15.937531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:15.939020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:16.013961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:16.029491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243abe-8198-4e5b-8293-b8989b9c15f0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:16.039511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:16.040940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:16.041870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:16.042482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:16.053881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:16.056266 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:16.069740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 929/1864] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:16.103931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:16.105896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:16.119544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e95cb-5b22-4b77-944b-156c1e72adb0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:16.126250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:16.127385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:16.128314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:16.128809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] PATH_INFO: `/v3/projects/7002666003b44a94a9feded3951a798b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:16.130324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7002666003b44a94a9feded3951a798b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:16.133021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:16.232573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7002666003b44a94a9feded3951a798b', 'request_id': 'req-a0ca4089-9725-4233-874b-d1fe2533f297'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.240222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7002666003b44a94a9feded3951a798b', 'request_id': 'req-a0ca4089-9725-4233-874b-d1fe2533f297'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.248746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7002666003b44a94a9feded3951a798b', 'request_id': 'req-a0ca4089-9725-4233-874b-d1fe2533f297'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.265578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] The token cache is being invalidate because project 7002666003b44a94a9feded3951a798b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:16.265876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.266802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.268392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 936/1865] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:16 2022] DELETE /identity/v3/projects/7002666003b44a94a9feded3951a798b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:16.273064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:16.283590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3979e-cc11-42e3-ae83-5cd1095f6ae2 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:16.355117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:16.356163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:16.356530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:16.356817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] PATH_INFO: `/v3/users/433f45c4db8046ab956240ed27d7a76a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:16.358025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=433f45c4db8046ab956240ed27d7a76a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:16.359992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:16.467474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433f45c4db8046ab956240ed27d7a76a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.478371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433f45c4db8046ab956240ed27d7a76a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.499388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433f45c4db8046ab956240ed27d7a76a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.501959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433f45c4db8046ab956240ed27d7a76a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.513349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433f45c4db8046ab956240ed27d7a76a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.522611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 930/1866] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:16 2022] DELETE /identity/v3/users/433f45c4db8046ab956240ed27d7a76a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:16.646036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:16.654487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:16.669915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca4089-9725-4233-874b-d1fe2533f297 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:16.731674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:16.732423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:16.732711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:16.732985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] PATH_INFO: `/v3/projects/3a962a4a42c1442ab39dc5280d2af86a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:16.734186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a962a4a42c1442ab39dc5280d2af86a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:16.736817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:16.790506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:16.798362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a962a4a42c1442ab39dc5280d2af86a', 'request_id': 'req-db161610-fb93-402d-b23c-b94170d00d25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.805293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a962a4a42c1442ab39dc5280d2af86a', 'request_id': 'req-db161610-fb93-402d-b23c-b94170d00d25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.813246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a962a4a42c1442ab39dc5280d2af86a', 'request_id': 'req-db161610-fb93-402d-b23c-b94170d00d25'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.827344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] The token cache is being invalidate because project 3a962a4a42c1442ab39dc5280d2af86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:16.827640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.828303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:16.829900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 937/1867] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:16 2022] DELETE /identity/v3/projects/3a962a4a42c1442ab39dc5280d2af86a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:16.834024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:16.836314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:16.849388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a37925-6733-49ea-b5a3-f5afd57e01d3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:16.862502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:16.864101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:16.954107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:16.955628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:16.955917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:16.956193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] PATH_INFO: `/v3/users/ffe8c2fe6625431f9f1073df35eb127b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:16.959029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe8c2fe6625431f9f1073df35eb127b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:16.963807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:16.971501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:16.976208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db161610-fb93-402d-b23c-b94170d00d25 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:16.982116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:16.982960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:16.983255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:16.983538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] PATH_INFO: `/v3/projects/0a6a2e9ab2174c90a93425ddb8cf2c61` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:16.985562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6a2e9ab2174c90a93425ddb8cf2c61)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:16.987959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:17.059056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe8c2fe6625431f9f1073df35eb127b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.074723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6a2e9ab2174c90a93425ddb8cf2c61', 'request_id': 'req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.075535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe8c2fe6625431f9f1073df35eb127b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.087302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6a2e9ab2174c90a93425ddb8cf2c61', 'request_id': 'req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.100974 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6a2e9ab2174c90a93425ddb8cf2c61', 'request_id': 'req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.109185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe8c2fe6625431f9f1073df35eb127b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.110428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe8c2fe6625431f9f1073df35eb127b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.129770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe8c2fe6625431f9f1073df35eb127b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.131515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] The token cache is being invalidate because project 0a6a2e9ab2174c90a93425ddb8cf2c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:17.132098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.133249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.135634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 938/1868] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:16 2022] DELETE /identity/v3/projects/0a6a2e9ab2174c90a93425ddb8cf2c61 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:17.136844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:17.139879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.144870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 931/1869] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:16 2022] DELETE /identity/v3/users/ffe8c2fe6625431f9f1073df35eb127b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:17.146095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:17.149006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.294277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a9763-306a-4be7-9f0f-8a08f800e7ff admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:17.295012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63b3aa-8e8d-494d-9fb7-0ec0dd794b5a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:17.318537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:17.318537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.318537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.318537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] PATH_INFO: `/v3/users/105795a0f85040c8b12f477967328d72` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.319157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105795a0f85040c8b12f477967328d72)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:17.319230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:17.319312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:17.319398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.319398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.319398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] PATH_INFO: `/v3/projects/1ab7522cbd6a4334934ea00ca327e497` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.319398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab7522cbd6a4334934ea00ca327e497)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:17.319398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:17.319690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:17.410923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105795a0f85040c8b12f477967328d72'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.423765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105795a0f85040c8b12f477967328d72'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.441052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab7522cbd6a4334934ea00ca327e497', 'request_id': 'req-0d359276-575b-43cd-a205-cf214689c95e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.452048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab7522cbd6a4334934ea00ca327e497', 'request_id': 'req-0d359276-575b-43cd-a205-cf214689c95e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.461274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab7522cbd6a4334934ea00ca327e497', 'request_id': 'req-0d359276-575b-43cd-a205-cf214689c95e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.464221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105795a0f85040c8b12f477967328d72'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.465344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105795a0f85040c8b12f477967328d72'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.475120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105795a0f85040c8b12f477967328d72'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.485154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 932/1870] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:17 2022] DELETE /identity/v3/users/105795a0f85040c8b12f477967328d72 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:17.486425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:17.489149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.489600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] The token cache is being invalidate because project 1ab7522cbd6a4334934ea00ca327e497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:17.489907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.491062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d359276-575b-43cd-a205-cf214689c95e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.495040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 939/1871] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:17 2022] DELETE /identity/v3/projects/1ab7522cbd6a4334934ea00ca327e497 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:17.498933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:17.500434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a7d73-c9b3-4bc7-a1b6-b27771d97684 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.500816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a7d73-c9b3-4bc7-a1b6-b27771d97684 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.501158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a7d73-c9b3-4bc7-a1b6-b27771d97684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.555236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6a5a7d73-c9b3-4bc7-a1b6-b27771d97684 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:17.561951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6a5a7d73-c9b3-4bc7-a1b6-b27771d97684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.632199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daadba7-0570-4891-9e07-a4b9dd954472 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:17.643164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:17.644549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.645676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.646236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] PATH_INFO: `/v3/projects/c2ca73dc54964f3fac41545b0bf21b61` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.649892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ca73dc54964f3fac41545b0bf21b61)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:17.657520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:17.717888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 940/1872] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:17.717888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57b6a-cdd6-4b1f-a1d5-f587dbcc6724 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.717888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57b6a-cdd6-4b1f-a1d5-f587dbcc6724 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.717888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57b6a-cdd6-4b1f-a1d5-f587dbcc6724 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.722867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 941/1873] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 12:59:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:59:17.724208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc3a3-e384-4f8f-9098-d55efd469fbd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.724575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc3a3-e384-4f8f-9098-d55efd469fbd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.724788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadc3a3-e384-4f8f-9098-d55efd469fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.766072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ca73dc54964f3fac41545b0bf21b61', 'request_id': 'req-95809ca6-62c6-411a-9cfe-3af366667831'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.784760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ca73dc54964f3fac41545b0bf21b61', 'request_id': 'req-95809ca6-62c6-411a-9cfe-3af366667831'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.786082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ccadc3a3-e384-4f8f-9098-d55efd469fbd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:17.788474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ccadc3a3-e384-4f8f-9098-d55efd469fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.799897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ca73dc54964f3fac41545b0bf21b61', 'request_id': 'req-95809ca6-62c6-411a-9cfe-3af366667831'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.822893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] The token cache is being invalidate because project c2ca73dc54964f3fac41545b0bf21b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:17.825225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.827540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95809ca6-62c6-411a-9cfe-3af366667831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:17.830404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 933/1874] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:17 2022] DELETE /identity/v3/projects/c2ca73dc54964f3fac41545b0bf21b61 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:17.831656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4800-eec4-439d-9700-a90adc88e59f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.831943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4800-eec4-439d-9700-a90adc88e59f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.832221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4800-eec4-439d-9700-a90adc88e59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.862002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-098f4800-eec4-439d-9700-a90adc88e59f None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:17.864788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-098f4800-eec4-439d-9700-a90adc88e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.896894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 942/1875] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:17.902081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.902427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.902710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.925548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:17.927115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:17.937628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 943/1876] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:17.938856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:17.949107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5503f-e251-4b25-a1a2-0f0d91ebbfa7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:17.957709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:17.958847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.959138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.959412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.960066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:17.960750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:17.963949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:17.964648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 934/1877] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:17.965992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:17.966412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:17.966752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:17.984185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 944/1878] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:17.985448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:17.988833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:17.990429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.001904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f14a5-f5be-4b8c-b4a9-b47b0504c5e7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.005511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 935/1879] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:18.006472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.007672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.008302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.008710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.009000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.009954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.013180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.018950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951ef4c-a657-4fb9-af8d-6d76aab642c4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.028676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:18.029161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.029861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.030259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.030540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.032700 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:18.033041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.035381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.078979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 945/1880] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:17 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:18.078979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.079205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 936/1881] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:18.080840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.084072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.088184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.103559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:18.107572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41f13c-ecc3-47fb-afbf-f33724ce192f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.172669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.173307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.173582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.173852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] PATH_INFO: `/v3/projects/3cd7e35b0fd8468c8b6128a90bdc8bc5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.175000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cd7e35b0fd8468c8b6128a90bdc8bc5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.176957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.177620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9065fa8-a93b-4133-99f0-11d1ddd0256a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.183530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.184216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.184603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.184963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.193923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.195489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.207168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 946/1882] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:18.208015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.218885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d39f9-5346-4b49-adf7-e336c5b8a5c1 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.233092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cd7e35b0fd8468c8b6128a90bdc8bc5', 'request_id': 'req-0a33523e-bac8-40f6-bf87-a372f69c64bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:18.239320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cd7e35b0fd8468c8b6128a90bdc8bc5', 'request_id': 'req-0a33523e-bac8-40f6-bf87-a372f69c64bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:18.245851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cd7e35b0fd8468c8b6128a90bdc8bc5', 'request_id': 'req-0a33523e-bac8-40f6-bf87-a372f69c64bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:18.262233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] The token cache is being invalidate because project 3cd7e35b0fd8468c8b6128a90bdc8bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:18.262597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:18.263423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:18.266118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 937/1883] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:18 2022] DELETE /identity/v3/projects/3cd7e35b0fd8468c8b6128a90bdc8bc5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:18.269198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.276897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.283678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02607e-fbe1-496d-9b81-e745448983ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.284323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02607e-fbe1-496d-9b81-e745448983ff admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.284648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02607e-fbe1-496d-9b81-e745448983ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.284880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02607e-fbe1-496d-9b81-e745448983ff admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.285504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02607e-fbe1-496d-9b81-e745448983ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.287413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02607e-fbe1-496d-9b81-e745448983ff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.309475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 947/1884] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:18.310737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.311031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.311311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.334445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:18.335962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.350732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 948/1885] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:18.350732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.350732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.354975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a33523e-bac8-40f6-bf87-a372f69c64bf admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.359494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74db092-ff46-4e81-a01a-2882f5748afb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.361830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.362455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.362901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.363284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.364064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.364229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0077f49-28f8-4c84-a9d6-336f0d009122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.364846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0077f49-28f8-4c84-a9d6-336f0d009122 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.365149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0077f49-28f8-4c84-a9d6-336f0d009122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.365410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0077f49-28f8-4c84-a9d6-336f0d009122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.366304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.366402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0077f49-28f8-4c84-a9d6-336f0d009122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.368002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0077f49-28f8-4c84-a9d6-336f0d009122 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.379525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 949/1886] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:18.380886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.381180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.381462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.403886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:18.405142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.406159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 938/1887] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:18.407107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.436527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:18.461205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:18.483800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 950/1888] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:18.484764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.487607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf62dd-774c-478e-aa21-ff8580836a74 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.492393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.493647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.494149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.494634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.495800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.497587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.508175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 939/1889] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:18.508453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2be95-094d-49f8-b1c7-1fdbeb5a0d7b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.509839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.516887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.517510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.517792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.518065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.518701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.520648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.522083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd14ea-e91b-4e2c-869d-b9673c4c39f7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.526630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.527216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.527497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.527765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.528414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:18.528716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.530280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.549276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 940/1890] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:18.549625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 951/1891] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/users => generated 293 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:18.550025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.550449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.560933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45dc2-c738-4b8d-a0da-9b0b2fdf354e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.562355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5c66-c0e4-4a1f-baed-8c73e2689679 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.565830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.566436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.566719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.566991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.567621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.567780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.568613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.569039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.569229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.569720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] PATH_INFO: `/v3/projects/afbfd0aa7639400dbde19d8b561e482e/users/cf772fb44a8f41bcb657311b0935006d/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.580766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 952/1892] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:18.581665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.589673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbfd0aa7639400dbde19d8b561e482e, user_id=cf772fb44a8f41bcb657311b0935006d, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.592260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.593757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb644-47f8-49ad-8bce-241b9f00ccb6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.598900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882e5bf-ff17-486b-8754-e2daafad572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.599594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882e5bf-ff17-486b-8754-e2daafad572a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.599865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882e5bf-ff17-486b-8754-e2daafad572a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.600163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882e5bf-ff17-486b-8754-e2daafad572a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.600835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882e5bf-ff17-486b-8754-e2daafad572a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.602803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882e5bf-ff17-486b-8754-e2daafad572a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.617200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 941/1893] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:18 2022] PUT /identity/v3/projects/afbfd0aa7639400dbde19d8b561e482e/users/cf772fb44a8f41bcb657311b0935006d/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:18.617200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.648578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 953/1894] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:18.650775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.651062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.651371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.707815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739be4c-9367-4aaf-aaac-fd4cd90d346f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.732969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] MFA Rules not processed for user `cf772fb44a8f41bcb657311b0935006d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:18.737468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:18.785458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.786460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.787281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.788446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.789324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.792404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.817055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 942/1895] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:18.818414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.840395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e69793-2627-4a1f-9351-fdbfd11feb93 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.843637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.843637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.843637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.843637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.844006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.846372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.880470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 943/1896] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/users => generated 292 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:18.892347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.902762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:18.907859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4451e4-ae5c-4d43-880c-355170223f9f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.915603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 954/1897] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:18.916909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.925666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.927427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.927891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.928449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] PATH_INFO: `/v3/projects/8eb19ba940cc4356a720b6591ae02b4f/users/2cc60497a0f44ca2918e1547946be28a/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.929295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0636d2-f288-44e1-a104-898a227494f8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:18.934653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:18.935431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:18.935813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:18.936351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:18.937102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.938857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.952872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb19ba940cc4356a720b6591ae02b4f, user_id=2cc60497a0f44ca2918e1547946be28a, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:18.958593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:18.975446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 955/1898] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:18.979717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:18.993564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 944/1899] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:18 2022] PUT /identity/v3/projects/8eb19ba940cc4356a720b6591ae02b4f/users/2cc60497a0f44ca2918e1547946be28a/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:18.995375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.004867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb2b06-8411-4a2f-aa59-bd70f379ca19 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.094217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282f413-0836-4dae-9d39-05a7ce40a288 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.101057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e89f6-7438-4307-82e3-2d82ee1d72a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.101880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e89f6-7438-4307-82e3-2d82ee1d72a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.103015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e89f6-7438-4307-82e3-2d82ee1d72a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.103892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e89f6-7438-4307-82e3-2d82ee1d72a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.104742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e89f6-7438-4307-82e3-2d82ee1d72a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.107176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e89f6-7438-4307-82e3-2d82ee1d72a9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.124311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 945/1900] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:19.126104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.126529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.126931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.141319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.142316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.143372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.144304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.146274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.153446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.190630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] MFA Rules not processed for user `2cc60497a0f44ca2918e1547946be28a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:19.194629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:19.230383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:19.252029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 956/1901] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:18 2022] POST /identity/v3/projects => generated 383 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:19.254201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.299351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 946/1902] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:19.300388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.316612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707329f-26e2-4594-ba95-1d3040d80859 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.417384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.418496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.418661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec571e6e-9d26-4a42-896a-071d9929b133 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.419307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.419847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.420963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.423374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.427103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.428108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.428411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.428781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] PATH_INFO: `/v3/projects/75d8030029c74a689ef418a37acfdd8e/users/26d783542f0a41eda4776870cd04c576/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.447109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d8030029c74a689ef418a37acfdd8e, user_id=26d783542f0a41eda4776870cd04c576, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.450020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:19.450671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.472189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 947/1903] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:19 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:19.472473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 957/1904] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:19 2022] PUT /identity/v3/projects/75d8030029c74a689ef418a37acfdd8e/users/26d783542f0a41eda4776870cd04c576/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:19.474524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.477910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.504664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10cf2b-1535-4661-b73d-e9b13bf1b6de admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.579431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce931280-461a-4c9d-b133-2594d110481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.580453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce931280-461a-4c9d-b133-2594d110481d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.580758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce931280-461a-4c9d-b133-2594d110481d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.581027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce931280-461a-4c9d-b133-2594d110481d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.581660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce931280-461a-4c9d-b133-2594d110481d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.583455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce931280-461a-4c9d-b133-2594d110481d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.594570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 948/1905] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:19.596137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.596359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.596641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.627393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] MFA Rules not processed for user `26d783542f0a41eda4776870cd04c576`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:19.628663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:19.637089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea013dc-7858-40de-8088-a96d129bbe33 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.642826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.643574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.643867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.644148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.644813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.647187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.694124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 958/1906] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:19 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:19.694124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.700050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 949/1907] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:19.704166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.716426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f7a91-d3c7-46f2-9cd4-293d87f23a65 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.762823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730dbc2-21ab-474f-97b1-8fe64ffd24e6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.767755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.768453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.768742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.769023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] PATH_INFO: `/v3/projects/f0243b53f19743b186d0e042c7f4474b/users/a15f22fc1e434316ad87c86b63be28b0/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.783639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:19.787617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0243b53f19743b186d0e042c7f4474b, user_id=a15f22fc1e434316ad87c86b63be28b0, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.790798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.793499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.794113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.794416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.794684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.795414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.797454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.805424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 959/1908] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:19 2022] PUT /identity/v3/projects/f0243b53f19743b186d0e042c7f4474b/users/a15f22fc1e434316ad87c86b63be28b0/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:19.806221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.822489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 950/1909] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:19 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:19.823533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.833574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f0b58-cbf4-4037-8ec1-5b34ba8d2d2d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.841335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:19.877240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5c24b-cccc-44bc-9d1f-82755ebe13f5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.883075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.883745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.884240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.884609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.885243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.887963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.898527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.899182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.899464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.899736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.900405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.902020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:19.914680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 951/1910] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:19.916301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.924058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 960/1911] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:19 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:19.924058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:19.927348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399d72f-b826-4028-ba44-243bbec503d7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:19.994181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:19.994943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:19.995290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:19.995569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:19.996491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:19.997858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.004579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769f8ea-4c68-449d-84b7-d280a13972ab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.010209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.010878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.011210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.011489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] PATH_INFO: `/v3/projects/f0243b53f19743b186d0e042c7f4474b/users/a15f22fc1e434316ad87c86b63be28b0/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.016879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 952/1912] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:20.017177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0243b53f19743b186d0e042c7f4474b, user_id=a15f22fc1e434316ad87c86b63be28b0, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.017255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.017334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.028459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65546f94-2479-463a-aaf3-43de5c6c01ad admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.029798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 961/1913] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:19 2022] PUT /identity/v3/projects/f0243b53f19743b186d0e042c7f4474b/users/a15f22fc1e434316ad87c86b63be28b0/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:20.030480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.090158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e72a1-7945-46b0-8464-c2a99c16c3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.090771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e72a1-7945-46b0-8464-c2a99c16c3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.091059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e72a1-7945-46b0-8464-c2a99c16c3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.091335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e72a1-7945-46b0-8464-c2a99c16c3e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.092646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e72a1-7945-46b0-8464-c2a99c16c3e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.097367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e72a1-7945-46b0-8464-c2a99c16c3e3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.097490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6303de9-47d0-4926-ac73-f52154106e11 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.101206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.101927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.102164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.102717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] PATH_INFO: `/v3/projects/26517af9d8934aeeb230d226d432c5bc/users/f90475fa90944f2bbd931fedc723a9a7/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.119599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 953/1914] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:20 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:20.119599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.119599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.119599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.119982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26517af9d8934aeeb230d226d432c5bc, user_id=f90475fa90944f2bbd931fedc723a9a7, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.122485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.124838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:20.138390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 962/1915] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:20 2022] PUT /identity/v3/projects/26517af9d8934aeeb230d226d432c5bc/users/f90475fa90944f2bbd931fedc723a9a7/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:20.139089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.147889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] MFA Rules not processed for user `a15f22fc1e434316ad87c86b63be28b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:20.149221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:20.168931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:20.203956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39992e2b-5468-4149-afac-53fb9d68fba7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.208602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.209208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.209490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.209758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.210387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.212773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.219857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 954/1916] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:20.220690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.223575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 963/1917] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:20.227111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.235397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06315443-d499-4f1c-b4de-66064e134fb9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.238014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182828a-8159-491a-8c4c-9377ad48a6c5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.240989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.241588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.241860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.242128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.242564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.242758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.243234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.243517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.243787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.244464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.244728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.246070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.255728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 955/1918] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:20.259490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 964/1919] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:20.259625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.260265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.260772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:20.270245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdd837-8fb1-4f1f-8f3c-dfa39ba44efe admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.275680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.276356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.276642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.276948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] PATH_INFO: `/v3/projects/26517af9d8934aeeb230d226d432c5bc/users/f90475fa90944f2bbd931fedc723a9a7/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.278885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26517af9d8934aeeb230d226d432c5bc, user_id=f90475fa90944f2bbd931fedc723a9a7, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.281534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.293888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 965/1920] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:20 2022] PUT /identity/v3/projects/26517af9d8934aeeb230d226d432c5bc/users/f90475fa90944f2bbd931fedc723a9a7/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:20.303019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.331902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23fa516-97a6-48b4-9580-97d2dbe69195 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.368653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6cc48-9c53-4b08-a07f-4f21d988af97 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.373572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60077578-12e4-4fff-94ba-ba3e06990125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.374253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60077578-12e4-4fff-94ba-ba3e06990125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.374538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60077578-12e4-4fff-94ba-ba3e06990125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.374835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60077578-12e4-4fff-94ba-ba3e06990125 admin admin] PATH_INFO: `/v3/projects/7024c81bb6ca45e8a814c469b13b1cb1/users/6bebb6a31bd741949f297466f634abc6/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.387751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a8381-9e05-4aec-8c16-705d82ff840f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.388406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a8381-9e05-4aec-8c16-705d82ff840f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.388694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a8381-9e05-4aec-8c16-705d82ff840f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.388968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a8381-9e05-4aec-8c16-705d82ff840f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.392980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60077578-12e4-4fff-94ba-ba3e06990125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7024c81bb6ca45e8a814c469b13b1cb1, user_id=6bebb6a31bd741949f297466f634abc6, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.397327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60077578-12e4-4fff-94ba-ba3e06990125 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.418206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 966/1921] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:20 2022] PUT /identity/v3/projects/7024c81bb6ca45e8a814c469b13b1cb1/users/6bebb6a31bd741949f297466f634abc6/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:20.419471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.419763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.420042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.449661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] MFA Rules not processed for user `f90475fa90944f2bbd931fedc723a9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:20.450929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:20.460230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a8381-9e05-4aec-8c16-705d82ff840f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.461679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a8381-9e05-4aec-8c16-705d82ff840f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.472416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 956/1922] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:20.473448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26665202-6684-4021-b762-be9cb7f9476f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.473733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26665202-6684-4021-b762-be9cb7f9476f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.474005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26665202-6684-4021-b762-be9cb7f9476f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.496885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:20.511984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-26665202-6684-4021-b762-be9cb7f9476f None None] MFA Rules not processed for user `6bebb6a31bd741949f297466f634abc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:20.513224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-26665202-6684-4021-b762-be9cb7f9476f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:20.518767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:20.578177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 967/1923] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:20.590754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.605707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 957/1924] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:20.608560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26665202-6684-4021-b762-be9cb7f9476f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.695748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ed55-22c4-4fe4-859e-adc512c1f566 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.702322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.703125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.703409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.703684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.706840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26665202-6684-4021-b762-be9cb7f9476f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.715032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.716505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.716791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.717062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.717701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.719615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.728360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.730199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.740860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 968/1925] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:20.778830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.787981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 958/1926] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:20 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:20.789970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.796223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24e2b-56d3-4557-80e5-ee2f54fcb08d service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.887411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5f816-ced6-4911-946f-156adecf01ee admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.889920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.890154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.890484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.890768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] PATH_INFO: `/v3/users/2cc60497a0f44ca2918e1547946be28a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.892234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc60497a0f44ca2918e1547946be28a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.894508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.894644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.895221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.896426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.897448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.898184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.900863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.939361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 959/1927] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:20 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:20.942966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:20.964222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908a79-3577-43fc-b895-dc090201ead2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:20.980148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:20.980805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:20.981211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:20.981492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:20.982167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:20.983699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:20.990160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc60497a0f44ca2918e1547946be28a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:21.011448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc60497a0f44ca2918e1547946be28a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:21.012526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 960/1928] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:21.017362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:21.031782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241dd2f-16f8-4587-910b-a49a784054b6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:21.032499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc60497a0f44ca2918e1547946be28a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:21.038441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:21.049459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:21.050300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.050598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.050797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] PATH_INFO: `/v3/projects/d01458cb54714b59ae476c24dba1017d/users/330b485fdcd3407dbe845fe613e7e3c6/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.057818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc60497a0f44ca2918e1547946be28a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:21.059116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9638729-78d5-438f-bf1a-0057a5e974d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc60497a0f44ca2918e1547946be28a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:21.065170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:21.070304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 969/1929] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:20 2022] DELETE /identity/v3/users/2cc60497a0f44ca2918e1547946be28a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:21.077079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.077553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.077980 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.083385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01458cb54714b59ae476c24dba1017d, user_id=330b485fdcd3407dbe845fe613e7e3c6, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:21.089355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:21.108139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:21.110714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:21.118780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 961/1930] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:21 2022] PUT /identity/v3/projects/d01458cb54714b59ae476c24dba1017d/users/330b485fdcd3407dbe845fe613e7e3c6/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:21.120676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:21.123385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:21.203837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 970/1931] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:21.212049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d347b68-d4f1-4e96-b4c1-8a077d286311 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:21.213520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:21.215005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:21.220598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:21.221255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.221539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.223753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.224526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:21.227855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:21.242403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 962/1932] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:21.244092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:21.255797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433a6c3-f070-4595-88f6-9636561e4b3c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:21.272685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582098e-f8e9-489f-9203-f776b4522fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:21.273435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582098e-f8e9-489f-9203-f776b4522fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.274284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582098e-f8e9-489f-9203-f776b4522fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.274644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582098e-f8e9-489f-9203-f776b4522fa7 admin admin] PATH_INFO: `/v3/projects/d01458cb54714b59ae476c24dba1017d/users/330b485fdcd3407dbe845fe613e7e3c6/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.276677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582098e-f8e9-489f-9203-f776b4522fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01458cb54714b59ae476c24dba1017d, user_id=330b485fdcd3407dbe845fe613e7e3c6, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:21.279424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582098e-f8e9-489f-9203-f776b4522fa7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:21.291874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 963/1933] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:21 2022] PUT /identity/v3/projects/d01458cb54714b59ae476c24dba1017d/users/330b485fdcd3407dbe845fe613e7e3c6/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:21.294114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4744-30e8-4380-b053-82c95a9bb9bf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:21.303801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.304096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.304405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.336252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] MFA Rules not processed for user `330b485fdcd3407dbe845fe613e7e3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:21.337693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:21.353539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:21.354457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.354747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.355082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.409156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 964/1934] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:21.420825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:21.423014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:21.424510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:21.424725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:21.435044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 971/1935] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:21.482270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ce968-a2b4-45e4-8f81-cd1a2ceb4d06 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:21.486102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:21.486715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.486998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.487268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.498242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:21.499690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:21.504021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:21.512446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 965/1936] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:21.901589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:21.904142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:21.916024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:21.927993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d32ed-f977-4976-9aa3-2e67d9de82b8 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:21.941136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:21.941803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:21.942086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:21.942360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] PATH_INFO: `/v3/projects/8eb19ba940cc4356a720b6591ae02b4f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:21.943516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb19ba940cc4356a720b6591ae02b4f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:21.945491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:22.024886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb19ba940cc4356a720b6591ae02b4f', 'request_id': 'req-09383323-6d16-4705-bbcc-8b7cb9624572'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.026323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb19ba940cc4356a720b6591ae02b4f', 'request_id': 'req-09383323-6d16-4705-bbcc-8b7cb9624572'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.035377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb19ba940cc4356a720b6591ae02b4f', 'request_id': 'req-09383323-6d16-4705-bbcc-8b7cb9624572'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.051189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] The token cache is being invalidate because project 8eb19ba940cc4356a720b6591ae02b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:22.051847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.052852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.056313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 972/1937] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:21 2022] DELETE /identity/v3/projects/8eb19ba940cc4356a720b6591ae02b4f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:22.059559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:22.073825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f4a37-ed48-4930-949a-e5e2fa5cd8ee service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:22.192158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:22.192834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:22.193111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:22.193534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] PATH_INFO: `/v3/users/f90475fa90944f2bbd931fedc723a9a7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:22.194670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90475fa90944f2bbd931fedc723a9a7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:22.196828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:22.259034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90475fa90944f2bbd931fedc723a9a7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.267630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90475fa90944f2bbd931fedc723a9a7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.294811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90475fa90944f2bbd931fedc723a9a7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.296476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90475fa90944f2bbd931fedc723a9a7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.304980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90475fa90944f2bbd931fedc723a9a7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:22.312949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 966/1938] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:22 2022] DELETE /identity/v3/users/f90475fa90944f2bbd931fedc723a9a7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:22.316418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:22.317823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:22.395443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383323-6d16-4705-bbcc-8b7cb9624572 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:22.487443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:22.488257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:22.488570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:22.489786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:22.507285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:22.612982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:22.616636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:22.619206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d64b78-842f-4d0d-a78b-a278822c6dc1 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:22.643414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 973/1939] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:23.526249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:23.533354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:23.599560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6bdbc-548a-472f-9ec6-edf1f8e6f81e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:23.604868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:23.605497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:23.605774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:23.606045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] PATH_INFO: `/v3/projects/26517af9d8934aeeb230d226d432c5bc` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:23.607269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26517af9d8934aeeb230d226d432c5bc)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:23.609200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:23.630236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:23.659182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26517af9d8934aeeb230d226d432c5bc', 'request_id': 'req-18eadd03-6768-4a32-b473-d6ba6e5f0060'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:23.666329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26517af9d8934aeeb230d226d432c5bc', 'request_id': 'req-18eadd03-6768-4a32-b473-d6ba6e5f0060'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:23.672799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26517af9d8934aeeb230d226d432c5bc', 'request_id': 'req-18eadd03-6768-4a32-b473-d6ba6e5f0060'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:23.686529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] The token cache is being invalidate because project 26517af9d8934aeeb230d226d432c5bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:23.686847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:23.687532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-18eadd03-6768-4a32-b473-d6ba6e5f0060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:23.689172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 967/1940] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:23 2022] DELETE /identity/v3/projects/26517af9d8934aeeb230d226d432c5bc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:23.694164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:23.694567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:23.694793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:23.715140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:23.716372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:23.777871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 974/1941] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:23.785852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:23.786122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:23.786420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:23.806756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:23.807951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:23.815718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 968/1942] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:23.820067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:23.830025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39230af-5b4b-42ff-81c4-8e76debd3e45 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:23.835068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:23.835888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:23.836231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:23.837068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:23.838154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:23.838154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:23.839769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:23.855578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 975/1943] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:23.860078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:23.869946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47b57a-3c75-4211-b675-6acd30b5b199 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:23.875541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:23.876192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:23.876965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:23.876965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:23.877438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:23.879422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:23.914125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 969/1944] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:23 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:23.916506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:23.929876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b8d92-4ac4-4624-8a3c-083c256b6183 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.016827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.017862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.018342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.018811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.019680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.022264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.053993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 976/1945] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:23 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:24.057987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.075338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10b240-55c4-4549-ac36-44f9349cd6a5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.116554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.116554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.117259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.117259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.118069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.119790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.131086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 970/1946] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:24.136158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.147748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd795-50ff-4a91-953e-4c73c33690ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.154582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6bdd3-539e-4a56-ba38-b4097279de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.155335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6bdd3-539e-4a56-ba38-b4097279de21 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.155644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6bdd3-539e-4a56-ba38-b4097279de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.155919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6bdd3-539e-4a56-ba38-b4097279de21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.156596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6bdd3-539e-4a56-ba38-b4097279de21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.158220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6bdd3-539e-4a56-ba38-b4097279de21 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.177327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 977/1947] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:24.178057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.193398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcae089-4cdc-46af-9792-a99bc77ea577 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.207978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.208691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.209016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.209295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] PATH_INFO: `/v3/projects/2cba2bb6ffd24cacb51128b07195bb97/users/0e74c36aa12043e4ac82cc228a6b9ccd/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.227670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cba2bb6ffd24cacb51128b07195bb97, user_id=0e74c36aa12043e4ac82cc228a6b9ccd, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.230580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.247964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 971/1948] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:24 2022] PUT /identity/v3/projects/2cba2bb6ffd24cacb51128b07195bb97/users/0e74c36aa12043e4ac82cc228a6b9ccd/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:24.259766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.260357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.260707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.296418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] MFA Rules not processed for user `0e74c36aa12043e4ac82cc228a6b9ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:24.297795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:24.360562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:24.378819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 978/1949] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:24.383925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.452961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f7ef3-4b1b-45ab-a8ff-3a3fd083630f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.457627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.458238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.458515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.458794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.459420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.461351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.489560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 972/1950] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:24 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:24.494525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.510555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d14584-70a9-4182-a489-de01820d3d82 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.591148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.592393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.592764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.593038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.593677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.595673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.622631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 979/1951] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:24 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:24.623514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.634056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b6dfe-c400-43d4-bd3a-48ce1efd16d5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.638900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.639508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.639869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.640049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.640721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.642580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.654038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 973/1952] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:24.658011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.668835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabbff8-4391-44f6-933c-d53ca22046f7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.673477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17506e62-9ca9-4af8-ace0-3d92ab64e94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.674153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506e62-9ca9-4af8-ace0-3d92ab64e94d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.674523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506e62-9ca9-4af8-ace0-3d92ab64e94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.674807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17506e62-9ca9-4af8-ace0-3d92ab64e94d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.675468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17506e62-9ca9-4af8-ace0-3d92ab64e94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.677108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17506e62-9ca9-4af8-ace0-3d92ab64e94d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.693807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 980/1953] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:24.698153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.710508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7f9e2-6687-42c1-84bb-cf8f6276ce07 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.716602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.717339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.717680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.718017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] PATH_INFO: `/v3/projects/8b572f4156194195938b51ffb8be19c0/users/c9b5d7b1ff6947feb136be29fe192e80/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.733527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b572f4156194195938b51ffb8be19c0, user_id=c9b5d7b1ff6947feb136be29fe192e80, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.736158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.738335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:24.754830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 974/1954] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:24 2022] PUT /identity/v3/projects/8b572f4156194195938b51ffb8be19c0/users/c9b5d7b1ff6947feb136be29fe192e80/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:24.765414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.765868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.765956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.796394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] MFA Rules not processed for user `c9b5d7b1ff6947feb136be29fe192e80`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:24.798095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:24.863683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 981/1955] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:24.868265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.930036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76b977-4cba-45dc-8a8f-24b93773a632 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:24.935199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:24.936217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:24.936217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:24.936432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:24.937033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:24.938960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:24.967292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 975/1956] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:24 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:24.971198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:24.983417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3012cf8-1c9c-4bd5-92ee-7f150ee31ec1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.049939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.050479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.050772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.051044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.051760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.053768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.062234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:25.078353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 982/1957] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:24 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:25.082521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.095660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a5593-397c-4965-badd-eb8d0ef5bd06 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.103377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.104000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.105785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.105785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.105785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.107047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.119221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 976/1958] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:25.123123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.142370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71aee8-8ace-47d0-95be-0c3f4560f687 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.147924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.148652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.148985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.150046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] PATH_INFO: `/v3/projects/b0f6418bf1054a47a90b18471943770b/users/57ef06bf7acc4aa6bb6a45343537be9f/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.176880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f6418bf1054a47a90b18471943770b, user_id=57ef06bf7acc4aa6bb6a45343537be9f, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.182988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.199520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 983/1959] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:25 2022] PUT /identity/v3/projects/b0f6418bf1054a47a90b18471943770b/users/57ef06bf7acc4aa6bb6a45343537be9f/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:25.203430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.225181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b5711-da87-4d96-a7d8-12332e05fe23 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.236321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.238757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:25.280062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.280717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.280999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.281396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.282066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.285595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.296607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 977/1960] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:25.300613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.311137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdffe1-6acf-42ed-a715-fa7c42c227b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.316461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f06b3-11ea-45e6-b674-1f40943d6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.317121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f06b3-11ea-45e6-b674-1f40943d6a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.317406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f06b3-11ea-45e6-b674-1f40943d6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.317824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f06b3-11ea-45e6-b674-1f40943d6a8f admin admin] PATH_INFO: `/v3/projects/b0f6418bf1054a47a90b18471943770b/users/57ef06bf7acc4aa6bb6a45343537be9f/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.319798 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f06b3-11ea-45e6-b674-1f40943d6a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f6418bf1054a47a90b18471943770b, user_id=57ef06bf7acc4aa6bb6a45343537be9f, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.322725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f06b3-11ea-45e6-b674-1f40943d6a8f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.326985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57518fb-54c6-4534-8c80-3c7ce20bcd93 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.332332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.333150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.333621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.334048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.336152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:25.354215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 978/1961] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:25 2022] PUT /identity/v3/projects/b0f6418bf1054a47a90b18471943770b/users/57ef06bf7acc4aa6bb6a45343537be9f/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:25.374539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.382695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.382695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.440039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] MFA Rules not processed for user `57ef06bf7acc4aa6bb6a45343537be9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:25.441702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:25.444093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.445757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.458682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 984/1962] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:25.492553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:25.554425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 979/1963] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:25.698385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.700477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:25.716218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd9381-41f1-463d-8a31-0f4ec97a9499 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.817331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.818617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.821392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.823284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:25.848742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:25.850496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:25.873676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 985/1964] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:25.966123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:25.986029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db2e53-0750-4fc2-b46e-59fbaea3dc6b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:25.994245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:25.994985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:25.995416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:25.995877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:26.123360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:26.125106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:26.136674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 980/1965] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:27.310385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:27.319395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b6b8e-3f96-40ec-82d0-287beed0fb1d service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:27.323835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:27.324575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:27.324749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:27.325047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:27.326773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:27.341325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:27.400709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:27.402173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcb527-e31c-43f7-820a-3a160bf70862 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:27.412054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 986/1966] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:29.822397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:29.823875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:29.911720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9444a1-a560-4ffe-8806-539f9faf4d06 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:29.916735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:29.917390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:29.917692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:29.917949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] PATH_INFO: `/v3/users/26d783542f0a41eda4776870cd04c576` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:29.919110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26d783542f0a41eda4776870cd04c576)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:29.921348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:29.979328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26d783542f0a41eda4776870cd04c576'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:29.988419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26d783542f0a41eda4776870cd04c576'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.011211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26d783542f0a41eda4776870cd04c576'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.011851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26d783542f0a41eda4776870cd04c576'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.015116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:30.032553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26d783542f0a41eda4776870cd04c576'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.048376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 981/1967] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:29 2022] DELETE /identity/v3/users/26d783542f0a41eda4776870cd04c576 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:30.050704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:30.051153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:30.051292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:30.088700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:30.090086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:30.155675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:30.157460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:30.174900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 987/1968] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:30.184851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:30.186225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:30.236539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ad953-bc9f-4b92-8406-a395361729d1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:30.241374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:30.242040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:30.242304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:30.242580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] PATH_INFO: `/v3/users/0e74c36aa12043e4ac82cc228a6b9ccd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:30.243948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e74c36aa12043e4ac82cc228a6b9ccd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:30.246038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:30.263114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34f7c9-8353-4097-9273-b1cb2254a5e6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:30.269150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ca6c6-b563-4a66-a63d-5a8f24c2eaa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:30.269969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ca6c6-b563-4a66-a63d-5a8f24c2eaa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:30.270255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ca6c6-b563-4a66-a63d-5a8f24c2eaa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:30.270532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ca6c6-b563-4a66-a63d-5a8f24c2eaa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:30.281685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ca6c6-b563-4a66-a63d-5a8f24c2eaa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:30.283375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ca6c6-b563-4a66-a63d-5a8f24c2eaa1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:30.299151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 988/1969] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:30.326020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e74c36aa12043e4ac82cc228a6b9ccd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.345487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e74c36aa12043e4ac82cc228a6b9ccd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.363199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e74c36aa12043e4ac82cc228a6b9ccd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.363932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e74c36aa12043e4ac82cc228a6b9ccd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.372992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e74c36aa12043e4ac82cc228a6b9ccd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:30.385237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 982/1970] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:30 2022] DELETE /identity/v3/users/0e74c36aa12043e4ac82cc228a6b9ccd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:30.386957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:30.387250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:30.387530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:30.409340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:30.410966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:30.499068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:30.513978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 989/1971] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:30.541955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:30.544307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:30.612407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4739dd7-5469-431c-a540-87e8f9649904 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:30.616555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:30.617287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:30.617639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:30.617987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:30.630146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:30.631774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:30.643542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 983/1972] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:30.948050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:30.949549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:30.962624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18df1a-8aa1-446f-8c80-03f01cdab770 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:30.967526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:30.968185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:30.969167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:30.969454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] PATH_INFO: `/v3/projects/75d8030029c74a689ef418a37acfdd8e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:30.970691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d8030029c74a689ef418a37acfdd8e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:30.972671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:31.035834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d8030029c74a689ef418a37acfdd8e', 'request_id': 'req-e8d4de85-154f-46cc-a127-5aa19e09a0ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.043394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d8030029c74a689ef418a37acfdd8e', 'request_id': 'req-e8d4de85-154f-46cc-a127-5aa19e09a0ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.063288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d8030029c74a689ef418a37acfdd8e', 'request_id': 'req-e8d4de85-154f-46cc-a127-5aa19e09a0ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.083891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] The token cache is being invalidate because project 75d8030029c74a689ef418a37acfdd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:31.084477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.085520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.088863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 990/1973] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:30 2022] DELETE /identity/v3/projects/75d8030029c74a689ef418a37acfdd8e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:31.093935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:31.114784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aae4e-3b48-47e5-8b84-3a1235fa3f39 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:31.131987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:31.179838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:31.182241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:31.211683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:31.213059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:31.213351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:31.213644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] PATH_INFO: `/v3/users/6bebb6a31bd741949f297466f634abc6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:31.215030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bebb6a31bd741949f297466f634abc6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:31.217597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:31.278622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bebb6a31bd741949f297466f634abc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.288664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bebb6a31bd741949f297466f634abc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.308072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bebb6a31bd741949f297466f634abc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.308854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bebb6a31bd741949f297466f634abc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.321030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bebb6a31bd741949f297466f634abc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.329686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d4de85-154f-46cc-a127-5aa19e09a0ac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:31.334768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 984/1974] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:31 2022] DELETE /identity/v3/users/6bebb6a31bd741949f297466f634abc6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:31.411070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:31.411738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:31.412022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:31.412342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] PATH_INFO: `/v3/projects/2cba2bb6ffd24cacb51128b07195bb97` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:31.413495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cba2bb6ffd24cacb51128b07195bb97)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:31.415695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:31.483881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:31.522138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cba2bb6ffd24cacb51128b07195bb97', 'request_id': 'req-34a47332-51f5-40c3-82cc-20d03fc5fb07'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.533001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cba2bb6ffd24cacb51128b07195bb97', 'request_id': 'req-34a47332-51f5-40c3-82cc-20d03fc5fb07'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.545592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cba2bb6ffd24cacb51128b07195bb97', 'request_id': 'req-34a47332-51f5-40c3-82cc-20d03fc5fb07'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.573183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] The token cache is being invalidate because project 2cba2bb6ffd24cacb51128b07195bb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:31.573574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.574196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.577446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 991/1975] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:31 2022] DELETE /identity/v3/projects/2cba2bb6ffd24cacb51128b07195bb97 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:31.581400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:31.590959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:31.606638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b581994-9705-488b-afbe-2ff66e427071 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:31.680969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:31.681695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:31.682279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:31.682279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] PATH_INFO: `/v3/users/c9b5d7b1ff6947feb136be29fe192e80` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:31.683724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b5d7b1ff6947feb136be29fe192e80)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:31.685568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:31.727399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:31.739559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b5d7b1ff6947feb136be29fe192e80'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.749073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b5d7b1ff6947feb136be29fe192e80'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.764520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b5d7b1ff6947feb136be29fe192e80'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.765317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b5d7b1ff6947feb136be29fe192e80'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.775358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b5d7b1ff6947feb136be29fe192e80'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:31.782674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 985/1976] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:31 2022] DELETE /identity/v3/users/c9b5d7b1ff6947feb136be29fe192e80 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:31.864558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:31.866365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:31.880356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a47332-51f5-40c3-82cc-20d03fc5fb07 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:31.966128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:31.968930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.005098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:32.006292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.006580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.006859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] PATH_INFO: `/v3/projects/7024c81bb6ca45e8a814c469b13b1cb1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.008127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7024c81bb6ca45e8a814c469b13b1cb1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:32.010753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:32.070995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22345981-6eee-48ce-a2fa-9dfcfb850112 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:32.099208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7024c81bb6ca45e8a814c469b13b1cb1', 'request_id': 'req-f5d36b26-f1c9-4c16-85ca-94ce314a815b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.106975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7024c81bb6ca45e8a814c469b13b1cb1', 'request_id': 'req-f5d36b26-f1c9-4c16-85ca-94ce314a815b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.114580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7024c81bb6ca45e8a814c469b13b1cb1', 'request_id': 'req-f5d36b26-f1c9-4c16-85ca-94ce314a815b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.141358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] The token cache is being invalidate because project 7024c81bb6ca45e8a814c469b13b1cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:32.142031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.143677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.145632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 992/1977] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:31 2022] DELETE /identity/v3/projects/7024c81bb6ca45e8a814c469b13b1cb1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:32.149917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:32.152895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.202841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:32.203533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.203764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.204036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] PATH_INFO: `/v3/users/cf772fb44a8f41bcb657311b0935006d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.205802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf772fb44a8f41bcb657311b0935006d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:32.208012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:32.271705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:32.276466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf772fb44a8f41bcb657311b0935006d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.287989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf772fb44a8f41bcb657311b0935006d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.296266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d36b26-f1c9-4c16-85ca-94ce314a815b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:32.300698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:32.307309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:32.307978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.308327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.308562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] PATH_INFO: `/v3/users/330b485fdcd3407dbe845fe613e7e3c6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.311891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=330b485fdcd3407dbe845fe613e7e3c6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:32.314059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:32.319511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf772fb44a8f41bcb657311b0935006d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.320641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf772fb44a8f41bcb657311b0935006d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.333238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf772fb44a8f41bcb657311b0935006d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.343811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 986/1978] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:31 2022] DELETE /identity/v3/users/cf772fb44a8f41bcb657311b0935006d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:32.344631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:32.346069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.386332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '330b485fdcd3407dbe845fe613e7e3c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.409812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '330b485fdcd3407dbe845fe613e7e3c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.418112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '330b485fdcd3407dbe845fe613e7e3c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.427407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9add-43d3-4d4f-a8a5-5cbc2be76ffc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:32.428288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '330b485fdcd3407dbe845fe613e7e3c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.428985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f15eb407-e449-4984-8140-dd5248392549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '330b485fdcd3407dbe845fe613e7e3c6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.433698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:32.434386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.434669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.434943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] PATH_INFO: `/v3/projects/8b572f4156194195938b51ffb8be19c0` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.436118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b572f4156194195938b51ffb8be19c0)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:32.440209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 993/1979] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:32 2022] DELETE /identity/v3/users/330b485fdcd3407dbe845fe613e7e3c6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:32.440398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:32.440482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.440482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.440482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.464333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:32.465558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.548661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b572f4156194195938b51ffb8be19c0', 'request_id': 'req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.555237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 994/1980] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:32.563403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b572f4156194195938b51ffb8be19c0', 'request_id': 'req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.573442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b572f4156194195938b51ffb8be19c0', 'request_id': 'req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.577620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:32.579496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.610995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] The token cache is being invalidate because project 8b572f4156194195938b51ffb8be19c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:32.611797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.614056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.617844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 987/1981] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:32 2022] DELETE /identity/v3/projects/8b572f4156194195938b51ffb8be19c0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:32.622087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:32.625677 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.698339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906da3b-f64b-421f-8d1e-93b756984135 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:32.758228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:32.765037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a220dd3-0f2a-439b-aa03-aa8668ddfb8c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:32.774249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:32.775867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.776221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.776670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] PATH_INFO: `/v3/users/57ef06bf7acc4aa6bb6a45343537be9f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.778299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ef06bf7acc4aa6bb6a45343537be9f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:32.780802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:32.805134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:32.843444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:32.844206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:32.844717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:32.845054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:32.852328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:32.867036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ef06bf7acc4aa6bb6a45343537be9f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.875123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:32.877114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:32.881651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ef06bf7acc4aa6bb6a45343537be9f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.894461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 995/1982] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:32.911265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ef06bf7acc4aa6bb6a45343537be9f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.913564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ef06bf7acc4aa6bb6a45343537be9f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.929525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4e9d5f31-5823-48c6-9791-afd60ab71b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ef06bf7acc4aa6bb6a45343537be9f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:32.940471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 988/1983] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:32 2022] DELETE /identity/v3/users/57ef06bf7acc4aa6bb6a45343537be9f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:33.093813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.095471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:33.119635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba14916-01e1-4833-b411-1e7f015fe202 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:33.262453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:33.263329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.263824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.264296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] PATH_INFO: `/v3/projects/d01458cb54714b59ae476c24dba1017d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.265830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d01458cb54714b59ae476c24dba1017d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:33.268922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:33.331212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd01458cb54714b59ae476c24dba1017d', 'request_id': 'req-032a788d-d760-406c-bc47-e81fc778760a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.339768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd01458cb54714b59ae476c24dba1017d', 'request_id': 'req-032a788d-d760-406c-bc47-e81fc778760a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.346808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd01458cb54714b59ae476c24dba1017d', 'request_id': 'req-032a788d-d760-406c-bc47-e81fc778760a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.362764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] The token cache is being invalidate because project d01458cb54714b59ae476c24dba1017d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:33.363333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.364623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-032a788d-d760-406c-bc47-e81fc778760a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.371643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 996/1984] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:33 2022] DELETE /identity/v3/projects/d01458cb54714b59ae476c24dba1017d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:33.373931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.374402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.374497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.406780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:33.408853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:33.503913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 989/1985] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:33.506578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.507054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.507440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.532442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:33.533959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:33.554699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 997/1986] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:33.564029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.579749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e666f-96f6-4f94-83cc-7b94d7ad11a1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:33.590028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:33.590627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.590912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.591186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.591990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:33.592317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:33.594250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:33.601634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.607195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:33.625165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 990/1987] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:33.626263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98269430-30f8-4d66-aa6f-ec65634269ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:33.630225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.642076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:33.642862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.643169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.643588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] PATH_INFO: `/v3/projects/afbfd0aa7639400dbde19d8b561e482e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.644726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afbfd0aa7639400dbde19d8b561e482e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:33.655676 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:33.666957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d623c-688b-4738-add2-2f269ebf3571 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:33.679404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:33.680248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.681673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.682076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.682847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:33.685192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:33.738923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:33.741467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afbfd0aa7639400dbde19d8b561e482e', 'request_id': 'req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.742351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 991/1988] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:33 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:33.743394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.752972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:33.754856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afbfd0aa7639400dbde19d8b561e482e', 'request_id': 'req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.765954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afbfd0aa7639400dbde19d8b561e482e', 'request_id': 'req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.784134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] The token cache is being invalidate because project afbfd0aa7639400dbde19d8b561e482e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:33.784512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.785438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:33.791375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 998/1989] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:33 2022] DELETE /identity/v3/projects/afbfd0aa7639400dbde19d8b561e482e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:33.791375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.793168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:33.870632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:33.878956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbbc38-b2c5-40fc-b7b1-f20d52f1eec5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:33.887167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:33.888669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.890481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.890986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] PATH_INFO: `/v3/projects/b0f6418bf1054a47a90b18471943770b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.892655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f6418bf1054a47a90b18471943770b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:33.895682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:33.914557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfea5ef-2f5e-4d3a-ac4f-7fb6f274bcf8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:33.929049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:33.929739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:33.940341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:33.940341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:33.940341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:33.942351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:33.985533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 999/1990] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:33 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:33.986820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:33.989076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.007227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d68263-ed5e-46f5-b92d-5a365cd0c721 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.044967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f6418bf1054a47a90b18471943770b', 'request_id': 'req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.062250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f6418bf1054a47a90b18471943770b', 'request_id': 'req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.078955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f6418bf1054a47a90b18471943770b', 'request_id': 'req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.131667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] The token cache is being invalidate because project b0f6418bf1054a47a90b18471943770b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:34.131880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.133766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.137360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 992/1991] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:33 2022] DELETE /identity/v3/projects/b0f6418bf1054a47a90b18471943770b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:34.138421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.142755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.161110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.162477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.163194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.163687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] PATH_INFO: `/v3/users/a15f22fc1e434316ad87c86b63be28b0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.166981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15f22fc1e434316ad87c86b63be28b0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.171513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.178622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a201ec-57ee-4e4c-8062-df8ad3088ef7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.190200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e691a10-af8f-402b-9246-47a467bfaff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.190835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691a10-af8f-402b-9246-47a467bfaff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.191117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691a10-af8f-402b-9246-47a467bfaff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.191393 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691a10-af8f-402b-9246-47a467bfaff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.192049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e691a10-af8f-402b-9246-47a467bfaff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.193929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e691a10-af8f-402b-9246-47a467bfaff2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.231882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 993/1992] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:34.233649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.234108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.234389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.253795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15f22fc1e434316ad87c86b63be28b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.270162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:34.270576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15f22fc1e434316ad87c86b63be28b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.271461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.273060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:34.280724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 994/1993] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:34.281702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.287549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15f22fc1e434316ad87c86b63be28b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.299027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15f22fc1e434316ad87c86b63be28b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.300194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97eb1e4b-955d-4474-ae31-e57a3dd931d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15f22fc1e434316ad87c86b63be28b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:34.300627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e472c-0e2a-48da-928b-b996776501fb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.306297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.306956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.307311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.307587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.308251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.312743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1000/1994] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:33 2022] DELETE /identity/v3/users/a15f22fc1e434316ad87c86b63be28b0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:34.312904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.313670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.313670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.313670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.335657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:34.337326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.344791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 995/1995] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:34.350610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.357597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.379294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5766e-f6ee-4066-955c-c588d410e147 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.433922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1001/1996] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:34.438066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.452569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f0e8b-f5eb-461f-9f39-6313e0b77a2c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.458663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.460422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.460422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.460422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.462118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:34.462118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.464946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.484494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1002/1997] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:34.488032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.515134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f08c6e-cca4-455c-9029-7f1fef5deade admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.520353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3f0c5-ac21-4be8-93ab-783b802585b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.521436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f0c5-ac21-4be8-93ab-783b802585b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.521936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f0c5-ac21-4be8-93ab-783b802585b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.522391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f0c5-ac21-4be8-93ab-783b802585b7 admin admin] PATH_INFO: `/v3/projects/69bd0d830d944e0fbe9a2a6214482a6f/users/a1a2f86009254c4181aa62da4248718d/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.538212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.539674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.541225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.543235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.544104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.548673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.564745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3f0c5-ac21-4be8-93ab-783b802585b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bd0d830d944e0fbe9a2a6214482a6f, user_id=a1a2f86009254c4181aa62da4248718d, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.568102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3f0c5-ac21-4be8-93ab-783b802585b7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.608748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 996/1998] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:34 2022] PUT /identity/v3/projects/69bd0d830d944e0fbe9a2a6214482a6f/users/a1a2f86009254c4181aa62da4248718d/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:34.611989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.612337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.612955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.626742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1003/1999] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:34 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:34.631299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.679908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] MFA Rules not processed for user `a1a2f86009254c4181aa62da4248718d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:34.682610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.726106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:34.795324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54160d1c-3499-4606-b4db-0e556d074c3a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.801086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 997/2000] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:34.804774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.806264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.806911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.807207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.808344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.809028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.810318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.829388 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ae96a-cb0f-4a33-9045-3bdaf9324eff None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.842487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.843144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1004/2001] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:34 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:34.845455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.846768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.847691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.848596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.849602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.851852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.859481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c33782-1f1b-4f26-a0b9-4da964816d15 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.870832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:34.871463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:34.871750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:34.872029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:34.872715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:34.874457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:34.897792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1005/2002] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:34.903339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.918926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 998/2003] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:34 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:34.921345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:34.925307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:34.925547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e722e-3580-4569-9247-6f6e00c825f8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:34.982200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:35.035583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024574e4-2440-4099-9188-2de2b7021540 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.043804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.044517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.044882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.045046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] PATH_INFO: `/v3/projects/f0243b53f19743b186d0e042c7f4474b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.046189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0243b53f19743b186d0e042c7f4474b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.048504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.067678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.069099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.069537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.069907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.070621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.073443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.089047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1006/2004] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:35.090164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.112364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085e30-35fa-43da-829d-592f96d07a35 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.117480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.118649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.119382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.119667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.120513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.122978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.153915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1007/2005] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/users => generated 300 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:35.154683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.159779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0243b53f19743b186d0e042c7f4474b', 'request_id': 'req-dc2e0f1d-6d15-464a-b038-a34eff48cc55'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:35.165134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc6057-8e7f-4c31-9b03-f6404ae37ff8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.169081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0243b53f19743b186d0e042c7f4474b', 'request_id': 'req-dc2e0f1d-6d15-464a-b038-a34eff48cc55'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:35.178018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0243b53f19743b186d0e042c7f4474b', 'request_id': 'req-dc2e0f1d-6d15-464a-b038-a34eff48cc55'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:35.196050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] The token cache is being invalidate because project f0243b53f19743b186d0e042c7f4474b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:35.197103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:35.198027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:35.199737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 999/2006] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:34 2022] DELETE /identity/v3/projects/f0243b53f19743b186d0e042c7f4474b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:35.204242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.205467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:35.250408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713d767-507c-45ca-a03e-f58738e32f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.251081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d767-507c-45ca-a03e-f58738e32f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.251364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d767-507c-45ca-a03e-f58738e32f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.251637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713d767-507c-45ca-a03e-f58738e32f2e admin admin] PATH_INFO: `/v3/projects/e289e877b7d84af5bf86d34151b533e8/users/3591d3a4a83b4902ba1122d5912a7ac8/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.271196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:35.284760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713d767-507c-45ca-a03e-f58738e32f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e289e877b7d84af5bf86d34151b533e8, user_id=3591d3a4a83b4902ba1122d5912a7ac8, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.287804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713d767-507c-45ca-a03e-f58738e32f2e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.297147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e0f1d-6d15-464a-b038-a34eff48cc55 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.302794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df3e6d-ff18-49ee-b278-f05aa78c611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.303568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3e6d-ff18-49ee-b278-f05aa78c611c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.303904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3e6d-ff18-49ee-b278-f05aa78c611c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.304184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df3e6d-ff18-49ee-b278-f05aa78c611c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.304881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3e6d-ff18-49ee-b278-f05aa78c611c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.306703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df3e6d-ff18-49ee-b278-f05aa78c611c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.323420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1008/2007] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:35 2022] PUT /identity/v3/projects/e289e877b7d84af5bf86d34151b533e8/users/3591d3a4a83b4902ba1122d5912a7ac8/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:35.327245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.327542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.328304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.342412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1000/2008] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:35.342412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d42db-0f52-4ac9-9c5d-ab94408129bc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.342412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d42db-0f52-4ac9-9c5d-ab94408129bc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.342412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d42db-0f52-4ac9-9c5d-ab94408129bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.361883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:35.363363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:35.375819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2a4d42db-0f52-4ac9-9c5d-ab94408129bc None None] MFA Rules not processed for user `3591d3a4a83b4902ba1122d5912a7ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:35.377275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2a4d42db-0f52-4ac9-9c5d-ab94408129bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:35.438969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1009/2009] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:35.439816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.449698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10979dde-e1bd-4993-9081-3e21392889e7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.454811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.455474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.455761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.456071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] PATH_INFO: `/v3/projects/ee02adee47a4454e803ac859017e29ae/users/f5ab09200b794ab0803369c0d18a84f4/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.466151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1001/2010] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:35.467518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.467813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.468091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.473077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee02adee47a4454e803ac859017e29ae, user_id=f5ab09200b794ab0803369c0d18a84f4, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.475813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.490239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:35.490913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1010/2011] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:35 2022] PUT /identity/v3/projects/ee02adee47a4454e803ac859017e29ae/users/f5ab09200b794ab0803369c0d18a84f4/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:35.491656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.491758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:35.494781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:35.504838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:35.560078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1002/2012] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:35.560933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.573218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cda22-0ad0-4bad-bdd0-23498b202c7d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.576513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab443e11-7e1a-4870-a851-2285d6b5b757 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.578829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.579450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.579748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.580012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.580694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.581218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.581830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.582113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.582385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.582560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.583181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.585111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.596167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1003/2013] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:35.596167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.608161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2eb68-db5e-4fb6-911d-d986273006a0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.613282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.613885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.614163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.614436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.615079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:35.615367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.617034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.620435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1011/2014] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:35.622064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.635751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1004/2015] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:35.636855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.647241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d095c7-dde7-47ae-a33a-1c5a44b5016f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.705147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:35.737290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.738544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.738544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.738723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.739297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.741789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.748634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b953-f7bf-4ce4-a6c9-53cb918475b6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.756249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.757372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.758040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.758495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] PATH_INFO: `/v3/projects/ee02adee47a4454e803ac859017e29ae/users/f5ab09200b794ab0803369c0d18a84f4/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.761833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee02adee47a4454e803ac859017e29ae, user_id=f5ab09200b794ab0803369c0d18a84f4, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.764830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.778037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1005/2017] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:35.778308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1012/2017] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:35 2022] PUT /identity/v3/projects/ee02adee47a4454e803ac859017e29ae/users/f5ab09200b794ab0803369c0d18a84f4/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:35.778948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.782185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.792847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa6708-300c-49ba-b9ce-fa9a7a10bd29 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.879428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbc485-b06a-49f3-9821-5c432fc72a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.880141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbc485-b06a-49f3-9821-5c432fc72a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.880544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbc485-b06a-49f3-9821-5c432fc72a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.880894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbc485-b06a-49f3-9821-5c432fc72a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.881614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbc485-b06a-49f3-9821-5c432fc72a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.885551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbc485-b06a-49f3-9821-5c432fc72a72 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.891065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bdabc-6983-4e23-a9ba-d56e47337be7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.901088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:35.903281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.903567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.903845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.904545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:35.921409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:35.925582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1006/2018] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:35.925582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:35.925582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:35.925582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:35.949455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1013/2019] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:35.950444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:35.962606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd02dbb6-555c-4fec-bc5e-a0e55963db43 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:35.974539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] MFA Rules not processed for user `f5ab09200b794ab0803369c0d18a84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:35.975762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:36.010629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:36.109215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.109965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.110257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.111726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.113585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.116872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.152571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1007/2020] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:36.152571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.168098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6683629-df55-4a21-afa6-39b5d7612537 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.174103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.175250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.175385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.175620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] PATH_INFO: `/v3/projects/3d53ff792aa644e9b6c5ccde431fe4a8/users/3a3d40992705484eaa958ccead469a3e/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.178185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1014/2021] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:35 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:36.179778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.182801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:36.207282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d53ff792aa644e9b6c5ccde431fe4a8, user_id=3a3d40992705484eaa958ccead469a3e, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.210322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.233332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1008/2022] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:36 2022] PUT /identity/v3/projects/3d53ff792aa644e9b6c5ccde431fe4a8/users/3a3d40992705484eaa958ccead469a3e/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:36.234574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.257985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:36.305890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e4f57-b4b1-4c13-898a-46e03b579162 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.359475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae074c-8352-48ec-aea9-4c584cb41975 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.371740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.371885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.372545 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.372825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.373127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.373247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.373668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.373948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.376311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.381772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.401663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1009/2023] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:36.405426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.439028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43edc92-6f6e-4d79-8f24-b62bcce525c6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.451098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.451724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.452495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.453756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.454426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.456171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.482852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1010/2024] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:36.485162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.492475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.495471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.510635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1015/2025] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:36.511947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159a117-9503-4906-bc97-1bfaa4225d24 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.512679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.524169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.526286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.526645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.527006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.527138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f92fcf-5e36-4932-af88-9536fca75596 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.527767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.529710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.532035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d839fa7-1f59-4680-8df2-a259aef66888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.532743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d839fa7-1f59-4680-8df2-a259aef66888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.533029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d839fa7-1f59-4680-8df2-a259aef66888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.533312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d839fa7-1f59-4680-8df2-a259aef66888 admin admin] PATH_INFO: `/v3/projects/3d53ff792aa644e9b6c5ccde431fe4a8/users/3a3d40992705484eaa958ccead469a3e/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.535452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d839fa7-1f59-4680-8df2-a259aef66888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d53ff792aa644e9b6c5ccde431fe4a8, user_id=3a3d40992705484eaa958ccead469a3e, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.538167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d839fa7-1f59-4680-8df2-a259aef66888 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.547151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1011/2026] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:36.548520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.561497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e8f39-12ba-4ddd-b13d-17bde2fb32db admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.563386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1016/2027] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:36 2022] PUT /identity/v3/projects/3d53ff792aa644e9b6c5ccde431fe4a8/users/3a3d40992705484eaa958ccead469a3e/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:36.579415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.579705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.579990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.604527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:36.651219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] MFA Rules not processed for user `3a3d40992705484eaa958ccead469a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:36.652639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:36.696155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa1a20-df5c-4fd7-9882-c68322e60763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:36.697204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa1a20-df5c-4fd7-9882-c68322e60763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.697562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa1a20-df5c-4fd7-9882-c68322e60763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.698685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa1a20-df5c-4fd7-9882-c68322e60763 admin admin] PATH_INFO: `/v3/projects/7d25a012f3b0443d9c971e3e115ed724/users/251202b31426496cb1d887b9a66932d5/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.732880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa1a20-df5c-4fd7-9882-c68322e60763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d25a012f3b0443d9c971e3e115ed724, user_id=251202b31426496cb1d887b9a66932d5, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:36.746704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa1a20-df5c-4fd7-9882-c68322e60763 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:36.785398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1012/2028] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:36 2022] PUT /identity/v3/projects/7d25a012f3b0443d9c971e3e115ed724/users/251202b31426496cb1d887b9a66932d5/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:36.793909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:36.794260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:36.794926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:36.811376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1017/2029] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:36.839358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:36.841682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:36.864003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] MFA Rules not processed for user `251202b31426496cb1d887b9a66932d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:36.865496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:36.988254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda61ce-f8d6-4e8e-afc1-66c7202b07be None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:36.998394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1013/2030] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:37.029247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:37.034012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:37.079488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:37.080209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:37.080585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:37.080955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:37.134855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a8c2-9aaa-4729-ad23-3f6462029946 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:37.142283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:37.143018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:37.143365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:37.143763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:37.153776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:37.155505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:37.157363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:37.160185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:37.161857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:37.166862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1014/2031] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:37.174344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1018/2032] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:37.414169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:37.430052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8303-f433-49de-bc72-0e52f1dacdf4 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:37.439370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb454e6-d6ea-4012-86f5-ea5cdeb67950 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:37.440413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb454e6-d6ea-4012-86f5-ea5cdeb67950 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:37.440760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb454e6-d6ea-4012-86f5-ea5cdeb67950 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:37.441311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb454e6-d6ea-4012-86f5-ea5cdeb67950 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:37.601876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb454e6-d6ea-4012-86f5-ea5cdeb67950 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:37.605746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb454e6-d6ea-4012-86f5-ea5cdeb67950 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:37.630638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1015/2033] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:37.933467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:38.016018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311980d-26b9-4603-9de3-534f576f1c3f service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:38.021365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:38.022083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:38.022587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:38.022873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] PATH_INFO: `/v3/users/a1a2f86009254c4181aa62da4248718d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:38.024008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a2f86009254c4181aa62da4248718d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:38.026189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:38.074827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a2f86009254c4181aa62da4248718d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.086905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a2f86009254c4181aa62da4248718d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.094594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a2f86009254c4181aa62da4248718d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.103867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a2f86009254c4181aa62da4248718d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.104675 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a2f86009254c4181aa62da4248718d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.112310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1019/2034] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:37 2022] DELETE /identity/v3/users/a1a2f86009254c4181aa62da4248718d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:38.116945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:38.117365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:38.117651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:38.140900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:38.142720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:38.220682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1016/2035] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:38.236347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:38.238466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:38.301729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc62f7-d82c-4ea9-8220-62b11fdc53c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:38.305453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:38.306603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:38.307024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:38.307300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:38.316981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:38.318408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:38.329784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1020/2036] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:38.782846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:38.784347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:38.800635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fda37-8283-498f-a38d-2a5a83ed1c75 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:38.807330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:38.808219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:38.808694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:38.809124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] PATH_INFO: `/v3/projects/69bd0d830d944e0fbe9a2a6214482a6f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:38.810447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69bd0d830d944e0fbe9a2a6214482a6f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:38.812915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:38.888521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:38.899064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69bd0d830d944e0fbe9a2a6214482a6f', 'request_id': 'req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.923433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69bd0d830d944e0fbe9a2a6214482a6f', 'request_id': 'req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.931031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69bd0d830d944e0fbe9a2a6214482a6f', 'request_id': 'req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.955183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] The token cache is being invalidate because project 69bd0d830d944e0fbe9a2a6214482a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:38.955475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.956122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:38.962760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:38.964395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1017/2037] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:38 2022] DELETE /identity/v3/projects/69bd0d830d944e0fbe9a2a6214482a6f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:38.973636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bd8df-535a-4168-a96f-0461e63a977a service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:38.988421 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:39.053733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:39.054465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.054764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.055038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] PATH_INFO: `/v3/users/f5ab09200b794ab0803369c0d18a84f4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.056260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ab09200b794ab0803369c0d18a84f4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:39.058282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:39.116021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ab09200b794ab0803369c0d18a84f4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.128109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ab09200b794ab0803369c0d18a84f4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.135481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ab09200b794ab0803369c0d18a84f4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.143821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ab09200b794ab0803369c0d18a84f4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.144509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-77348937-8628-46d9-a803-c42d4a3a2e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ab09200b794ab0803369c0d18a84f4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.151050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1021/2038] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:38 2022] DELETE /identity/v3/users/f5ab09200b794ab0803369c0d18a84f4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:39.433637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:39.435486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:39.456405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead80fb-ffb3-4dd4-ae20-7b36ad4d3488 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:39.546563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:39.547523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.548145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.548145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] PATH_INFO: `/v3/projects/ee02adee47a4454e803ac859017e29ae` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.549287 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee02adee47a4454e803ac859017e29ae)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:39.551870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:39.618416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee02adee47a4454e803ac859017e29ae', 'request_id': 'req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.626385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee02adee47a4454e803ac859017e29ae', 'request_id': 'req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.632941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee02adee47a4454e803ac859017e29ae', 'request_id': 'req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.651887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] The token cache is being invalidate because project ee02adee47a4454e803ac859017e29ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:39.652200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.652893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7c6754e6-b29e-4d7d-b6c8-6b0be2479f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:39.654509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1018/2039] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:39 2022] DELETE /identity/v3/projects/ee02adee47a4454e803ac859017e29ae => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:39.659451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.659765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.660041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.680636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:39.681934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:39.750422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1022/2040] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:39.759430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.759749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.760024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.781604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:39.783285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:39.793494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1019/2041] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:39.799865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:39.811397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31caa6-0537-473e-9ed7-49dc6d98440b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:39.816671 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:39.817261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.817539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.817872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.818642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:39.818848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:39.820440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:39.833408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1023/2042] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:39.838846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:39.848315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb6fb7-6699-4fe8-aa4b-f38141c7f87f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:39.853147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:39.853741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.854019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.854286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.854909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:39.856849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:39.882978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1020/2043] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:39 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:39.887597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:39.897863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036a7d-064c-41c2-ab04-b3695e2affd2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:39.918566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:39.964766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:39.965463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:39.965816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:39.966636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:39.967167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:39.969626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:39.999165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1024/2044] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:39 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:40.000469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.012491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ab60c-e06c-4e71-b294-a9fde94fb50f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.017628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.018326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.018680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.019024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.019748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.021582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.037823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1021/2045] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:40.039097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.049619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0b1e5-a309-4cab-9b97-83a7dff1bce7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.054448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.057344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.057636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.057914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.058834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.060856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.093006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1025/2046] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:40.096639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.103398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.106171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:40.119176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8349-0714-44c0-bc22-a181e00e3b45 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.128109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28c8af-6158-4aac-b131-164996f015e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.128992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28c8af-6158-4aac-b131-164996f015e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.129425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28c8af-6158-4aac-b131-164996f015e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.129844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28c8af-6158-4aac-b131-164996f015e8 admin admin] PATH_INFO: `/v3/projects/5845f920233e4302838ce169f197eb48/users/2d4e24436e1f4d65a3b8d3964f0a54a6/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.137307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6ba0-6849-44bb-a12a-caf38ae92cf5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.154419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28c8af-6158-4aac-b131-164996f015e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5845f920233e4302838ce169f197eb48, user_id=2d4e24436e1f4d65a3b8d3964f0a54a6, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.158634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.159293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.159574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.159847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] PATH_INFO: `/v3/users/251202b31426496cb1d887b9a66932d5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.161031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=251202b31426496cb1d887b9a66932d5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.162940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.168972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28c8af-6158-4aac-b131-164996f015e8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.172219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:40.199741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1022/2047] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:40 2022] PUT /identity/v3/projects/5845f920233e4302838ce169f197eb48/users/2d4e24436e1f4d65a3b8d3964f0a54a6/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:40.215139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.215475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.215699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.236705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '251202b31426496cb1d887b9a66932d5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:40.264465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '251202b31426496cb1d887b9a66932d5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:40.264840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] MFA Rules not processed for user `2d4e24436e1f4d65a3b8d3964f0a54a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:40.264840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:40.268796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '251202b31426496cb1d887b9a66932d5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:40.281499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '251202b31426496cb1d887b9a66932d5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:40.282350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-879509fe-fc8b-4c59-9f9c-9aaa23ffe460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '251202b31426496cb1d887b9a66932d5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:40.291697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1026/2048] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:40 2022] DELETE /identity/v3/users/251202b31426496cb1d887b9a66932d5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:40.297054 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.297417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.297706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.328002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:40.329717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:40.353465 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1023/2049] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:40.358905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.369671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:40.411424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1027/2050] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:40.428957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.431198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:40.484635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312cbfd-b222-4abe-88fb-7faae05ce84e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.484635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.484635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.484635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.485105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.485458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.490215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.490369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:40.529154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1024/2051] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:40 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:40.533396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.548631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d6d8-6265-40ca-a0f1-5a66fc113a5e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.619751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc17b05-f17f-4f72-b6ad-e21d1cb234cc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.619998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.620492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.620776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.621643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.627821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.629151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.629623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.630005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.630707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.631082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.632551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.633164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.645017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1028/2052] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:40.652712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:40.661476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1025/2053] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:40 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:40.665747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.678534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc001d6-d40a-44ed-bc11-835deb69544a service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.685531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.685828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.686043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.686429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.687065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.690360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.703143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1029/2054] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:40.707673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.719957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714413fb-8b09-4f4c-8109-18ce210a3a76 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.725438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.726276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.726488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.726766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] PATH_INFO: `/v3/projects/aca298ccf68c42088da646c81c87f2a2/users/83685475b7ea4cab9ae384028cd340ac/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.750584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca298ccf68c42088da646c81c87f2a2, user_id=83685475b7ea4cab9ae384028cd340ac, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.753651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.777216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.778278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1026/2055] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:40 2022] PUT /identity/v3/projects/aca298ccf68c42088da646c81c87f2a2/users/83685475b7ea4cab9ae384028cd340ac/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:40.791371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79412d29-5519-4fad-8984-d8ec4cde478a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.874023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22481a1-bd5d-46dc-8284-5029449cb135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.875130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22481a1-bd5d-46dc-8284-5029449cb135 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.875130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22481a1-bd5d-46dc-8284-5029449cb135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.875335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22481a1-bd5d-46dc-8284-5029449cb135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.875964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22481a1-bd5d-46dc-8284-5029449cb135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.877926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22481a1-bd5d-46dc-8284-5029449cb135 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.908914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1030/2056] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:40.913952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:40.950037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904840d0-ebd1-4fb4-9e2e-c9f254e9277a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:40.955456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:40.957613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.957811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.958177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] PATH_INFO: `/v3/projects/aca298ccf68c42088da646c81c87f2a2/users/83685475b7ea4cab9ae384028cd340ac/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.960770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca298ccf68c42088da646c81c87f2a2, user_id=83685475b7ea4cab9ae384028cd340ac, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:40.964971 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:40.981780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1027/2057] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:40 2022] PUT /identity/v3/projects/aca298ccf68c42088da646c81c87f2a2/users/83685475b7ea4cab9ae384028cd340ac/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:40.995500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:40.995500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:40.995500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:40.999314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.000866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.034032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] MFA Rules not processed for user `83685475b7ea4cab9ae384028cd340ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:41.035445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.082960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9174fa8-7a04-43c6-80c0-a2621434cf5e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.090086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.090774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.091057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.091333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] PATH_INFO: `/v3/projects/7d25a012f3b0443d9c971e3e115ed724` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.092606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d25a012f3b0443d9c971e3e115ed724)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.094580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.104577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:41.130386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1031/2058] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:41.142823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.144450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.154065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d25a012f3b0443d9c971e3e115ed724', 'request_id': 'req-588c624d-4418-4465-b0ce-9c4721723ea9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:41.161110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d25a012f3b0443d9c971e3e115ed724', 'request_id': 'req-588c624d-4418-4465-b0ce-9c4721723ea9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:41.168996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d25a012f3b0443d9c971e3e115ed724', 'request_id': 'req-588c624d-4418-4465-b0ce-9c4721723ea9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:41.206335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] The token cache is being invalidate because project 7d25a012f3b0443d9c971e3e115ed724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:41.206631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:41.208247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-588c624d-4418-4465-b0ce-9c4721723ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:41.211093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1028/2059] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:40 2022] DELETE /identity/v3/projects/7d25a012f3b0443d9c971e3e115ed724 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:41.215989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccda4a-add2-4814-bfda-4db6df200f9c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.216316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccda4a-add2-4814-bfda-4db6df200f9c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.216597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccda4a-add2-4814-bfda-4db6df200f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.223938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9047e9-c553-4d07-a3d5-ab29f660a303 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.239211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4cccda4a-add2-4814-bfda-4db6df200f9c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:41.240585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4cccda4a-add2-4814-bfda-4db6df200f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.292605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:41.296724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.297353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.297635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.297906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.300380 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.314782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1029/2060] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:41.323417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.323822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.324096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.347379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:41.348631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.358361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1030/2061] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:41.363337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.373964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70523-1b0e-4700-a731-264e061b050c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.376391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.377891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.378652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.379291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.379577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.379853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.380528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:41.380837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.382612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.389172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1032/2062] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:41.403855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1031/2063] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:59:41.404979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.418783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0517a35-8229-404a-bbbe-124907260933 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.426601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.427490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.428435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.428789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.429505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.431845 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.466465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1033/2064] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:41 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:41.470818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.482051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a6d57-624d-416c-a130-1c5e23ade117 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.577619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.578369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.578646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.578919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.579587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.581708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.620424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1032/2065] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:41 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:41.632801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.645541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da571d-e83a-4eb0-a7a7-9834d6f3f79c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.651707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.652594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.653027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.653516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.654305 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.656095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.669717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1034/2066] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:41.673918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.688395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94de23d-deb9-49e5-9d80-f2c1bfd1f8b6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.696322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9330c7f-4ea7-4a8f-b4df-4cd5f3feb9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.697152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9330c7f-4ea7-4a8f-b4df-4cd5f3feb9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.697616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9330c7f-4ea7-4a8f-b4df-4cd5f3feb9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.697978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9330c7f-4ea7-4a8f-b4df-4cd5f3feb9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.698769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9330c7f-4ea7-4a8f-b4df-4cd5f3feb9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.700533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9330c7f-4ea7-4a8f-b4df-4cd5f3feb9a4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.714127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1033/2067] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:41.718514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.729766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60fa61-a077-4202-98b0-3f12e0983ff8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:41.742658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:41.744869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.745466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.745900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] PATH_INFO: `/v3/projects/c32a7fa3267240989a0384ea463dd212/users/c75ef7d06792467b8bf5cf0f40824c2d/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.770070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32a7fa3267240989a0384ea463dd212, user_id=c75ef7d06792467b8bf5cf0f40824c2d, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:41.775227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:41.792126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:41.801410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1035/2068] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:41 2022] PUT /identity/v3/projects/c32a7fa3267240989a0384ea463dd212/users/c75ef7d06792467b8bf5cf0f40824c2d/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:41.813579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:41.813864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:41.814149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:41.855530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] MFA Rules not processed for user `c75ef7d06792467b8bf5cf0f40824c2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:41.857967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:41.969217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1034/2069] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:41.983374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:41.985326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:42.135012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc114261-2514-4224-8339-f64603b8891d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:42.139343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178bd49-a063-4904-a0fa-491c73f78da3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:42.140184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178bd49-a063-4904-a0fa-491c73f78da3 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:42.140651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178bd49-a063-4904-a0fa-491c73f78da3 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:42.141028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178bd49-a063-4904-a0fa-491c73f78da3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:42.151762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178bd49-a063-4904-a0fa-491c73f78da3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:42.153474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178bd49-a063-4904-a0fa-491c73f78da3 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:42.163572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1036/2070] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:44.123518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:44.125361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:44.202596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:44.237619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed2881-157b-49f0-b5dd-8f7351053da9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:44.243320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:44.243965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:44.244422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:44.244725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] PATH_INFO: `/v3/users/2d4e24436e1f4d65a3b8d3964f0a54a6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:44.245881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4e24436e1f4d65a3b8d3964f0a54a6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:44.247943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:44.325459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4e24436e1f4d65a3b8d3964f0a54a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:44.338819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4e24436e1f4d65a3b8d3964f0a54a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:44.364161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4e24436e1f4d65a3b8d3964f0a54a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:44.364956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4e24436e1f4d65a3b8d3964f0a54a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:44.374113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4e24436e1f4d65a3b8d3964f0a54a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:44.381344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1035/2071] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:44 2022] DELETE /identity/v3/users/2d4e24436e1f4d65a3b8d3964f0a54a6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:44.386902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:44.387474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:44.387474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:44.420075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:44.421507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:44.505325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1037/2072] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:44.515870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:44.517631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:44.578867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec3031-ee22-42f3-bd8e-dd4ec70926ec admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:44.582405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:44.583013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:44.583308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:44.583578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:44.593360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:44.594805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:44.607397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1036/2073] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:44.889026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:44.892515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:44.931564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c8b91-d68f-42a4-b384-935c46b2cb8d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:44.936431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:44.937131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:44.937411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:44.937685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] PATH_INFO: `/v3/projects/5845f920233e4302838ce169f197eb48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:44.938825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5845f920233e4302838ce169f197eb48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:44.940848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:44.989076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:44.990444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.004891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5845f920233e4302838ce169f197eb48', 'request_id': 'req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.006547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34f347-0d08-4767-90dd-377474aed15e service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.013900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5845f920233e4302838ce169f197eb48', 'request_id': 'req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.023324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5845f920233e4302838ce169f197eb48', 'request_id': 'req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.057578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] The token cache is being invalidate because project 5845f920233e4302838ce169f197eb48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:45.058130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.058803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.060360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1038/2074] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:44 2022] DELETE /identity/v3/projects/5845f920233e4302838ce169f197eb48 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:45.064308 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.065910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.073178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:45.089541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:45.091055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.091900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.092038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] PATH_INFO: `/v3/users/3591d3a4a83b4902ba1122d5912a7ac8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.093365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3591d3a4a83b4902ba1122d5912a7ac8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:45.095576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:45.099235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:45.161883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ea37c-c9d0-4158-a774-b909f6a16f9d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.163877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3591d3a4a83b4902ba1122d5912a7ac8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.167426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:45.168095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.168410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.168690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] PATH_INFO: `/v3/users/83685475b7ea4cab9ae384028cd340ac` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.169889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83685475b7ea4cab9ae384028cd340ac)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:45.171259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3591d3a4a83b4902ba1122d5912a7ac8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.171819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:45.183115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3591d3a4a83b4902ba1122d5912a7ac8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.183818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3591d3a4a83b4902ba1122d5912a7ac8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.191603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ffb1ca84-415a-40d9-b5a6-ab3ea2ea762b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3591d3a4a83b4902ba1122d5912a7ac8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.199201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1037/2075] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:44 2022] DELETE /identity/v3/users/3591d3a4a83b4902ba1122d5912a7ac8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:45.204245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.204695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.204978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.229754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:45.230656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83685475b7ea4cab9ae384028cd340ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.230986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.246463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83685475b7ea4cab9ae384028cd340ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.253682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83685475b7ea4cab9ae384028cd340ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.262316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83685475b7ea4cab9ae384028cd340ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.262965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83685475b7ea4cab9ae384028cd340ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.269379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1039/2076] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:45 2022] DELETE /identity/v3/users/83685475b7ea4cab9ae384028cd340ac => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:45.297050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1038/2077] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:45.309020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.310519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.378950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85bd9-2266-4f1a-81f8-441dddc39d9e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.383269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfb479-7032-4f8b-9caf-c2d4c823f58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:45.384543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfb479-7032-4f8b-9caf-c2d4c823f58e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.384927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfb479-7032-4f8b-9caf-c2d4c823f58e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.385199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfb479-7032-4f8b-9caf-c2d4c823f58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.448623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:45.466203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfb479-7032-4f8b-9caf-c2d4c823f58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:45.467988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfb479-7032-4f8b-9caf-c2d4c823f58e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:45.478734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1040/2078] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:45.619731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.622878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.640629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f20a6-f3a4-430f-9072-94605afa765c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.646696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:45.647433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.647782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.648303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] PATH_INFO: `/v3/projects/aca298ccf68c42088da646c81c87f2a2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.650157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca298ccf68c42088da646c81c87f2a2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:45.652064 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:45.712768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca298ccf68c42088da646c81c87f2a2', 'request_id': 'req-44b4ae4b-bc6e-484b-a567-d69825469171'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.723208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca298ccf68c42088da646c81c87f2a2', 'request_id': 'req-44b4ae4b-bc6e-484b-a567-d69825469171'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.755910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca298ccf68c42088da646c81c87f2a2', 'request_id': 'req-44b4ae4b-bc6e-484b-a567-d69825469171'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.780794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] The token cache is being invalidate because project aca298ccf68c42088da646c81c87f2a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:45.781250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.781932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.783653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1039/2079] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:45 2022] DELETE /identity/v3/projects/aca298ccf68c42088da646c81c87f2a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:45.789063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb2806-32cd-4c07-97c4-e3cf0721af25 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.789439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb2806-32cd-4c07-97c4-e3cf0721af25 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.790288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb2806-32cd-4c07-97c4-e3cf0721af25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.813912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c0fb2806-32cd-4c07-97c4-e3cf0721af25 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:45.815138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c0fb2806-32cd-4c07-97c4-e3cf0721af25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.872010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.873538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.882062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1041/2080] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:45.883720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:45.891348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.892042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.892246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.893771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4ae4b-bc6e-484b-a567-d69825469171 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.906935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:45.907663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.908009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.908399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] PATH_INFO: `/v3/projects/e289e877b7d84af5bf86d34151b533e8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.909775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e289e877b7d84af5bf86d34151b533e8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:45.911745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:45.913435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:45.914505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:45.923115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1042/2081] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:45.927407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.937880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82dc8-d08a-4c28-bea5-e583910d1086 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.943898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:45.944631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:45.944911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:45.945181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:45.945843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:45.946280 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:45.948060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:45.964962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1043/2082] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:59:45.965169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e289e877b7d84af5bf86d34151b533e8', 'request_id': 'req-f0e9dba7-8888-4e53-98e8-f2253bc819a2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.966060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.968770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e289e877b7d84af5bf86d34151b533e8', 'request_id': 'req-f0e9dba7-8888-4e53-98e8-f2253bc819a2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.975780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e289e877b7d84af5bf86d34151b533e8', 'request_id': 'req-f0e9dba7-8888-4e53-98e8-f2253bc819a2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.976074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4b10-3f8c-4077-8a17-49d1d8f2790a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:45.990891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] The token cache is being invalidate because project e289e877b7d84af5bf86d34151b533e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:45.991532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.992301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:45.994952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1040/2083] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:45 2022] DELETE /identity/v3/projects/e289e877b7d84af5bf86d34151b533e8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:45.998538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:45.999942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.038116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.038726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.039010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.039285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.040487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.042430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.067922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9dba7-8888-4e53-98e8-f2253bc819a2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.072582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1044/2084] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:45 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:46.074982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.075793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.076410 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.076783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.077028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] PATH_INFO: `/v3/users/3a3d40992705484eaa958ccead469a3e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.078370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.078499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3d40992705484eaa958ccead469a3e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.080704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.148575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:46.151883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3d40992705484eaa958ccead469a3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.160380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3d40992705484eaa958ccead469a3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.174565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024cbb0-2bf7-41a5-b379-ae09a4181235 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.175635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3d40992705484eaa958ccead469a3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.176188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3d40992705484eaa958ccead469a3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.180494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.181327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.181793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.182075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.182936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.185009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3d40992705484eaa958ccead469a3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.185173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.191516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1041/2085] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 12:59:45 2022] DELETE /identity/v3/users/3a3d40992705484eaa958ccead469a3e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:46.209957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1045/2086] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:46 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:46.214939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.216599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.226849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:46.286688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2608a-bbeb-473a-a4a8-5cd3f7141bd0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.292138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.292682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.292964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.293230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.293876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.295613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.309904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1042/2087] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:46.311211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.322061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35149fc0-aa7a-4445-bdf1-a8bb8597feb9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.327255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4729e-3b29-44ff-87b0-69d1ee3118f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.327922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4729e-3b29-44ff-87b0-69d1ee3118f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.328453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4729e-3b29-44ff-87b0-69d1ee3118f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.328656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4729e-3b29-44ff-87b0-69d1ee3118f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.329382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4729e-3b29-44ff-87b0-69d1ee3118f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.330996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4729e-3b29-44ff-87b0-69d1ee3118f8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.343617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1046/2088] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:46.347214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.359067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658aa1d-478a-4d4f-b28f-cb780ae17666 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.363823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.364503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.364801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.365100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] PATH_INFO: `/v3/projects/127991e9e2a04562b68a481df832c646/users/803a2290376d4333bdec73aa902e73b5/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.382196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127991e9e2a04562b68a481df832c646, user_id=803a2290376d4333bdec73aa902e73b5, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.392616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.408861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1043/2089] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:46 2022] PUT /identity/v3/projects/127991e9e2a04562b68a481df832c646/users/803a2290376d4333bdec73aa902e73b5/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:46.422593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.423441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.423441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.454774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] MFA Rules not processed for user `803a2290376d4333bdec73aa902e73b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:46.456080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.524586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1047/2090] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:46.529190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.594557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444e818-5656-4dda-90c0-be2db9411634 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.599220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.599830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.600112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.600419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.601060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.603641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.639788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1044/2091] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:46 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:46.644202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.654212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.657661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38a6bb-6fd2-4a14-82dd-8ad21f9b4e06 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.659190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.734456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e15b2-bff2-4c0c-9b93-c4337dd903ac admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.738828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.739451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.739736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.740211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.740406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.740799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.740937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.741030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.741338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] PATH_INFO: `/v3/projects/3d53ff792aa644e9b6c5ccde431fe4a8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.742499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d53ff792aa644e9b6c5ccde431fe4a8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.743256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.744674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.751155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:46.768309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1048/2092] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:46 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:46.770699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.782168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ca924-38b0-4246-8d4f-1d4e81616fc7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.788061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.788707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.788983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.789256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.789985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.792452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.801881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d53ff792aa644e9b6c5ccde431fe4a8', 'request_id': 'req-e3e001be-7600-412c-b1a9-4a92563afa5f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.804251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1049/2093] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:46.808502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.811337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d53ff792aa644e9b6c5ccde431fe4a8', 'request_id': 'req-e3e001be-7600-412c-b1a9-4a92563afa5f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.817693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d53ff792aa644e9b6c5ccde431fe4a8', 'request_id': 'req-e3e001be-7600-412c-b1a9-4a92563afa5f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.818677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adda696-89b4-49c3-84fe-278decff12f6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.832731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] The token cache is being invalidate because project 3d53ff792aa644e9b6c5ccde431fe4a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:59:46.833022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.833730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e3e001be-7600-412c-b1a9-4a92563afa5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:59:46.835242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1045/2094] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 12:59:46 2022] DELETE /identity/v3/projects/3d53ff792aa644e9b6c5ccde431fe4a8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:46.841763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01fede-b86a-4470-a58d-367ce50d3a98 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.842128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01fede-b86a-4470-a58d-367ce50d3a98 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.842612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01fede-b86a-4470-a58d-367ce50d3a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.854771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:46.864486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5d01fede-b86a-4470-a58d-367ce50d3a98 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:46.865815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d01fede-b86a-4470-a58d-367ce50d3a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.898519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.899819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.900105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.900407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.901065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.902688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.920790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1050/2095] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:46.924916 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.926933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.935062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1046/2096] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:46.942491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.942774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.943050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.945507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac9b5-2ea4-4da0-a680-def095d1af90 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.954311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee1b00-24db-46d5-968d-81097ba80db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.955021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee1b00-24db-46d5-968d-81097ba80db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.955479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee1b00-24db-46d5-968d-81097ba80db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.955757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee1b00-24db-46d5-968d-81097ba80db9 admin admin] PATH_INFO: `/v3/projects/52952206c10c495d9f46af20826bde9f/users/7b833bd25f5449f98d8c3bb12a29edae/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.964800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:46.966034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:46.973890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1047/2097] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:46.978439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:46.978803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee1b00-24db-46d5-968d-81097ba80db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52952206c10c495d9f46af20826bde9f, user_id=7b833bd25f5449f98d8c3bb12a29edae, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.981542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee1b00-24db-46d5-968d-81097ba80db9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:46.987865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323da1c4-996f-43ad-ab57-7ab093d94743 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:46.993899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:46.994744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:46.995052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:46.995329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:46.995973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:59:46.996306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:46.996920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1051/2098] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:46 2022] PUT /identity/v3/projects/52952206c10c495d9f46af20826bde9f/users/7b833bd25f5449f98d8c3bb12a29edae/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:46.997902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.009058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.009345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.009619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.013010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1048/2099] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 12:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:59:47.017864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.033115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a37b34-3308-4945-b3f9-6599f89e7806 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.054842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] MFA Rules not processed for user `7b833bd25f5449f98d8c3bb12a29edae`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:47.056591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:47.105612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.106391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.106687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.107034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.107757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.109818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.127644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1052/2100] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:47.133028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.134624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:47.147326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1049/2101] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/projects => generated 391 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:47.150396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763556e-5545-42c5-8ee2-6669cc3f8479 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.151872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.215894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.216557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.216862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.217153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.217797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.219741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.220824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0defb69f-4484-4841-87ec-3084c1208833 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.226300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.226908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.227272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.227477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.228113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.230060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.248093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1053/2102] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:47.252156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.253478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1050/2103] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:59:47.257366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.266426 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:47.269015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12573551-0220-41d2-a0fc-4e4d28ab6633 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.326658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dec4f-9ae3-4b08-938a-23dfb7637f79 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.331462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.332186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.332522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.332802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.333439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.335361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.338190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.338887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.339216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.339582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.340219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.342005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.353366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1051/2104] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:47.357444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.361876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1054/2105] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:59:47.362987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.367781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14741b86-95ee-4212-8d17-bc311402ab90 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.373055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.373655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.373777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4c946-79c4-45c2-a2ca-316650b624d2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.373965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.374240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.375032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.376972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.379377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.380067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.380416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.380727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.381598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.383342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.387717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1052/2106] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:47.392117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.400353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1055/2107] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:59:47.401793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.404628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728ca8b-9662-452f-8093-9a0a0d4d0a18 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.409841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499461d5-47e7-45d5-8ed2-6c3f869e8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.410170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499461d5-47e7-45d5-8ed2-6c3f869e8aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.410454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499461d5-47e7-45d5-8ed2-6c3f869e8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.410739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499461d5-47e7-45d5-8ed2-6c3f869e8aeb admin admin] PATH_INFO: `/v3/projects/a207888f7afd4c66839de9f9c5da63c4/users/6b8f012a6f3f45ca84781897b5f539e0/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.413365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bb63e-5653-48fd-b87b-4d828fef5004 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.419700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.420587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.421041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.421469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] PATH_INFO: `/v3/projects/c78cade25b0044e0a258740b1500ce72/users/9f91c46ca942428289c018f9ba0bcb14/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.427143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499461d5-47e7-45d5-8ed2-6c3f869e8aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a207888f7afd4c66839de9f9c5da63c4, user_id=6b8f012a6f3f45ca84781897b5f539e0, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.430099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499461d5-47e7-45d5-8ed2-6c3f869e8aeb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.439162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78cade25b0044e0a258740b1500ce72, user_id=9f91c46ca942428289c018f9ba0bcb14, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.442377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.445328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1053/2108] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:47 2022] PUT /identity/v3/projects/a207888f7afd4c66839de9f9c5da63c4/users/6b8f012a6f3f45ca84781897b5f539e0/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:47.455498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.455873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.456258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.457528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1056/2109] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:47 2022] PUT /identity/v3/projects/c78cade25b0044e0a258740b1500ce72/users/9f91c46ca942428289c018f9ba0bcb14/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:59:47.461098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.487413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] MFA Rules not processed for user `6b8f012a6f3f45ca84781897b5f539e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:47.488678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:47.532981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc4a23-1359-4dd4-8da3-cabbd8c9af12 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.537964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.538577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.538861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.539136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.540376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.541401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:47.542039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.556222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1057/2110] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 12:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:59:47.560690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.571383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a5e00-7195-40fc-b4db-56ac3c1d9b4c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.573152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1054/2111] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:59:47.577074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee195d4-40a5-4966-80aa-815fcf5da538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.578195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee195d4-40a5-4966-80aa-815fcf5da538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.578195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee195d4-40a5-4966-80aa-815fcf5da538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.578835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee195d4-40a5-4966-80aa-815fcf5da538 admin admin] PATH_INFO: `/v3/projects/c78cade25b0044e0a258740b1500ce72/users/9f91c46ca942428289c018f9ba0bcb14/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.580686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee195d4-40a5-4966-80aa-815fcf5da538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78cade25b0044e0a258740b1500ce72, user_id=9f91c46ca942428289c018f9ba0bcb14, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.583584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee195d4-40a5-4966-80aa-815fcf5da538 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.585527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.592959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:47.594814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1058/2112] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 12:59:47 2022] PUT /identity/v3/projects/c78cade25b0044e0a258740b1500ce72/users/9f91c46ca942428289c018f9ba0bcb14/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:59:47.666428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.667972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.667972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.686813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91fd1f-d157-4ee2-8bd6-0ca8fdbddafd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.697837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] MFA Rules not processed for user `9f91c46ca942428289c018f9ba0bcb14`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:59:47.699153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:47.743931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.744575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.744848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.745381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.748042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:47.783239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1059/2113] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:59:47.795758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:47.797755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:47.811766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.813272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.814579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb367-2f07-4e1c-9a5a-96ab3c31f475 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:47.818826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:47.819614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:47.820102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:47.820752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:47.823470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:59:47.823604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1055/2114] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:47.896377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:47.897839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:47.908479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1060/2115] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:59:50.714024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:50.723230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7d70b-6371-47eb-ba74-4d23b6418c4e service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:50.726861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:50.728397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:50.728681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:50.728958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:50.778846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:59:50.791078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:50.792578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:50.802283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1056/2116] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:59:51.618946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:59:51.628567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23be8-8c5c-4070-b057-89c970b8a310 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:59:51.632329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:59:51.632996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:59:51.634358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:59:51.634511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:59:51.643968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:59:51.645437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:59:51.654986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1061/2117] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 12:59:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:00.797929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:01.650750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=45 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:02.690822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:02.692328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:02.759830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25747f-810c-4382-8555-17d24d5c7c1d service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:02.764837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:02.765495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:02.765901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:02.766180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] PATH_INFO: `/v3/users/082e5aa56ac147e6a8f8c797b269ff23` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:02.767968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=082e5aa56ac147e6a8f8c797b269ff23)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:02.769923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:02.819023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082e5aa56ac147e6a8f8c797b269ff23'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:02.826932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082e5aa56ac147e6a8f8c797b269ff23'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:02.841580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082e5aa56ac147e6a8f8c797b269ff23'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:02.842420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082e5aa56ac147e6a8f8c797b269ff23'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:02.851333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-02e3d896-a401-4b75-a478-153800b16a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082e5aa56ac147e6a8f8c797b269ff23'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:02.860309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1057/2118] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:02 2022] DELETE /identity/v3/users/082e5aa56ac147e6a8f8c797b269ff23 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:03.038786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.052607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32d384-d802-4fcc-b08e-2b021dd607f4 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.128092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.128092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.128092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.128092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] PATH_INFO: `/v3/projects/dc2cd7163bd9476891dfe836246ead22` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.129132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2cd7163bd9476891dfe836246ead22)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.131038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.180699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2cd7163bd9476891dfe836246ead22', 'request_id': 'req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:03.188256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2cd7163bd9476891dfe836246ead22', 'request_id': 'req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:03.196087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2cd7163bd9476891dfe836246ead22', 'request_id': 'req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:03.209498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] The token cache is being invalidate because project dc2cd7163bd9476891dfe836246ead22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:03.209815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:03.210424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84c223ef-40d2-44e1-ad20-5e0c4dedf1f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:03.211902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1062/2119] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:03 2022] DELETE /identity/v3/projects/dc2cd7163bd9476891dfe836246ead22 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:03.222710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.223006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.223315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.242755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:03.243978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:03.305226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1058/2120] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:03.313358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.313721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.314062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.335786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:03.337805 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:03.352585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1063/2121] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:03.356829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.367109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5078-140f-4f27-ab2e-f7728ff9a57e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.372523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.373149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.373456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.373707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.374412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:00:03.374715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.376468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.388679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1059/2122] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:00:03.393557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.404307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342c622-5bc7-4fc5-ae34-196575bd29ea None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.410408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.411343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.411684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.411890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.412563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.414515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.445817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1064/2123] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:03.447945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.458121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9da5f-e072-45e4-a4cf-85b1360f1fcb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.461623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:03.520372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.520999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.521277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.521547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.522187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.524106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.547020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1060/2124] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:03.551195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.562190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bca622-b4a2-45ae-9845-2b40e98d49b8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.566920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.567520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.567800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.568069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.568750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.570328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.582217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1065/2125] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:03.585916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.596027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4fd84-4a68-4704-8164-ed42b13aa6ae admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.600680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d546bc8-f5e7-406d-ae01-d3d1a2bcfd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.601430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d546bc8-f5e7-406d-ae01-d3d1a2bcfd3a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.601555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d546bc8-f5e7-406d-ae01-d3d1a2bcfd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.601834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d546bc8-f5e7-406d-ae01-d3d1a2bcfd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.602474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d546bc8-f5e7-406d-ae01-d3d1a2bcfd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.604797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d546bc8-f5e7-406d-ae01-d3d1a2bcfd3a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.615180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1061/2126] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:03.622469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.632659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485ab9b-9f8f-4cfe-b5e7-ac63c020060b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.637976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.638652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.638932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.639206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] PATH_INFO: `/v3/projects/9acf990959ad408b994e61aa8904bc33/users/600cb7eb3bf743fc911ed4eb2d4232e9/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.653885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9acf990959ad408b994e61aa8904bc33, user_id=600cb7eb3bf743fc911ed4eb2d4232e9, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.656562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.669324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1066/2127] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:03 2022] PUT /identity/v3/projects/9acf990959ad408b994e61aa8904bc33/users/600cb7eb3bf743fc911ed4eb2d4232e9/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:03.679524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.679813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.680137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.708142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] MFA Rules not processed for user `600cb7eb3bf743fc911ed4eb2d4232e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:03.709370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:03.776006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1062/2128] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:03.780859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.815388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:03.849127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2539-3bd0-4864-988e-e983f78c06e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.854315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.854914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.855196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.855497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.856124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.858058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.883620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1067/2129] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:00:03.887817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.898801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb736bb-e4af-4215-8246-8ffb4fa1af9a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:03.955237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:03.955857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:03.956144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:03.956453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:03.957103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:03.959071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:03.984047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1063/2130] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:00:03 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:03.985449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:03.996093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719472d-62bc-46a7-88c3-c91a980344f8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:04.000796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:04.001400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.001680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.001950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.002611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:04.004188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:04.018833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1068/2131] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:04.020433 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:04.027000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:04.031358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf37b4-b55d-46a7-b192-f0a13ac4b795 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:04.037085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:04.038472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.038760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.039041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] PATH_INFO: `/v3/projects/350c883e330b44d99d48f45404e000ee/users/b39e24527094484bbfaef5ae2e419268/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.055370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350c883e330b44d99d48f45404e000ee, user_id=b39e24527094484bbfaef5ae2e419268, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:04.058673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:04.073321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1064/2132] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:04 2022] PUT /identity/v3/projects/350c883e330b44d99d48f45404e000ee/users/b39e24527094484bbfaef5ae2e419268/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:04.077631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:04.090690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dad7d0-136f-4ca8-b1ad-ae29df2223be admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:04.162386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c152f85-6cd3-439a-add8-cb9d3a07b199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:04.163076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152f85-6cd3-439a-add8-cb9d3a07b199 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.163422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152f85-6cd3-439a-add8-cb9d3a07b199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.163758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c152f85-6cd3-439a-add8-cb9d3a07b199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.164817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c152f85-6cd3-439a-add8-cb9d3a07b199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:04.167237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c152f85-6cd3-439a-add8-cb9d3a07b199 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:04.179193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1069/2133] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:04.183654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:04.202458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a172b0-373c-43e7-9d3e-ec10f1613bc8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:04.208876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:04.210190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.210422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.210706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] PATH_INFO: `/v3/projects/350c883e330b44d99d48f45404e000ee/users/b39e24527094484bbfaef5ae2e419268/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.214190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=350c883e330b44d99d48f45404e000ee, user_id=b39e24527094484bbfaef5ae2e419268, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:04.217024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:04.229795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1065/2134] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:04 2022] PUT /identity/v3/projects/350c883e330b44d99d48f45404e000ee/users/b39e24527094484bbfaef5ae2e419268/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:00:04.242672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.242961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.243278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.275434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] MFA Rules not processed for user `b39e24527094484bbfaef5ae2e419268`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:04.276709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:04.364076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1070/2135] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:04.376882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:04.378609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:04.435995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e795d12-492e-49e6-9cc3-d445961f9a51 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:04.439593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104df3d-2785-4594-9e8b-df06de6d6159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:04.440199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104df3d-2785-4594-9e8b-df06de6d6159 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.440520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104df3d-2785-4594-9e8b-df06de6d6159 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.440796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104df3d-2785-4594-9e8b-df06de6d6159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.450507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104df3d-2785-4594-9e8b-df06de6d6159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:04.452438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104df3d-2785-4594-9e8b-df06de6d6159 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:04.461972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1066/2136] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:04.892241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:04.917627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:04.958435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaafea-aa06-4155-a7f0-468397d1f741 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:04.963285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:04.963935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:04.964219 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:04.964551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] PATH_INFO: `/v3/users/600cb7eb3bf743fc911ed4eb2d4232e9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:04.965742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] RBAC: Authorizing `identity:delete_user(user_id=600cb7eb3bf743fc911ed4eb2d4232e9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:04.967679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:05.018119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600cb7eb3bf743fc911ed4eb2d4232e9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.031224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600cb7eb3bf743fc911ed4eb2d4232e9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.039146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600cb7eb3bf743fc911ed4eb2d4232e9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.047434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600cb7eb3bf743fc911ed4eb2d4232e9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.048189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600cb7eb3bf743fc911ed4eb2d4232e9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.054591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1071/2137] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:04 2022] DELETE /identity/v3/users/600cb7eb3bf743fc911ed4eb2d4232e9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:05.059833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:05.060409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:05.060523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:05.079834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:05.081059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:05.141724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1067/2138] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:05.152711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:05.154225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:05.216301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace4d9-c152-49b5-b09e-32784e33a12e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:05.220345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:05.221147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:05.221503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:05.221844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:05.233244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:05.234759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:05.245866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1072/2139] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:05.531107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:05.532697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:05.545112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d1c3d-12a1-4f6e-b220-db54acac353b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:05.549696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:05.550334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:05.550615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:05.550893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] PATH_INFO: `/v3/projects/9acf990959ad408b994e61aa8904bc33` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:05.552245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9acf990959ad408b994e61aa8904bc33)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:05.554766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:05.602397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9acf990959ad408b994e61aa8904bc33', 'request_id': 'req-5646e099-4ca3-4173-934f-630fd8733408'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.607981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9acf990959ad408b994e61aa8904bc33', 'request_id': 'req-5646e099-4ca3-4173-934f-630fd8733408'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.613935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9acf990959ad408b994e61aa8904bc33', 'request_id': 'req-5646e099-4ca3-4173-934f-630fd8733408'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.616435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:05.627777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] The token cache is being invalidate because project 9acf990959ad408b994e61aa8904bc33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:05.628060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.628697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.630162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1068/2140] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:05 2022] DELETE /identity/v3/projects/9acf990959ad408b994e61aa8904bc33 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:05.635097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:05.645098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d807f-e637-4c80-ab64-b7acc9988202 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:05.686970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:05.703654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:05.704301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:05.704596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:05.704871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] PATH_INFO: `/v3/users/b39e24527094484bbfaef5ae2e419268` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:05.705998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b39e24527094484bbfaef5ae2e419268)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:05.707913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:05.755867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b39e24527094484bbfaef5ae2e419268'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.768090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b39e24527094484bbfaef5ae2e419268'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.775438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b39e24527094484bbfaef5ae2e419268'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.782715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b39e24527094484bbfaef5ae2e419268'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.783337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02d8f234-05a3-4a1e-a90c-2f376a00bffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b39e24527094484bbfaef5ae2e419268'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:05.789159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1073/2141] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:05 2022] DELETE /identity/v3/users/b39e24527094484bbfaef5ae2e419268 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:06.307339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.308934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:06.323033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e099-4ca3-4173-934f-630fd8733408 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.383245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.383957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.384483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.384595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] PATH_INFO: `/v3/projects/350c883e330b44d99d48f45404e000ee` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.389035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=350c883e330b44d99d48f45404e000ee)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.391110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.440999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350c883e330b44d99d48f45404e000ee', 'request_id': 'req-803c6ed3-1d42-479a-975d-69460d34e97f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:06.447274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350c883e330b44d99d48f45404e000ee', 'request_id': 'req-803c6ed3-1d42-479a-975d-69460d34e97f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:06.454315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350c883e330b44d99d48f45404e000ee', 'request_id': 'req-803c6ed3-1d42-479a-975d-69460d34e97f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:06.467482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] The token cache is being invalidate because project 350c883e330b44d99d48f45404e000ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:06.467794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:06.468466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-803c6ed3-1d42-479a-975d-69460d34e97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:06.469978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1069/2142] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:06 2022] DELETE /identity/v3/projects/350c883e330b44d99d48f45404e000ee => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:06.474824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.475285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.475371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.495986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:06.497204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:06.557819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1074/2143] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:00:06.565597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.565937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.566217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.586209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:06.587329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:06.595148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1070/2144] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:06.599258 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.609596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6b75b-1c04-4479-8187-d51a6a6003bb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.614165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.614768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.615050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.615352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.615994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:00:06.616338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.617947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.630234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1075/2145] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:00:06.635057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.644798 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a165e-01f8-46c2-af68-4778165d8bfe None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.649420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.650257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.650538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.650840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.651479 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.653431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.680242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1071/2146] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:00:06 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:06.685156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.695017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bfeeb-908c-4873-b7ce-bbb41400a443 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.744521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:06.755625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.756235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.756556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.756837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.757474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.760264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.780565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1076/2147] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:00:06 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:06.784619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.796129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7c5e3-f38b-4a63-a8f1-15c19eaca92d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.800686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.801320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.801600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.801876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.802545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.804137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.814940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1072/2148] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:06.818708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.828968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf2bb6-b174-47f8-99c7-d162a05af41d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.833485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de80c6-0481-4f14-8f4b-5c64b22ed180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.834072 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de80c6-0481-4f14-8f4b-5c64b22ed180 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.834351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de80c6-0481-4f14-8f4b-5c64b22ed180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.834622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de80c6-0481-4f14-8f4b-5c64b22ed180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.835243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de80c6-0481-4f14-8f4b-5c64b22ed180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.836909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de80c6-0481-4f14-8f4b-5c64b22ed180 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.848970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1077/2149] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:06.852376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:06.862832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27effe4c-32b5-42c3-a1e6-f3aaf46e048e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:06.867443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:06.868083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.869080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.869357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] PATH_INFO: `/v3/projects/6e02c27e6c5a4d318b46bb767599ee8b/users/c920088084d849a4866b5e50593b9003/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.884658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e02c27e6c5a4d318b46bb767599ee8b, user_id=c920088084d849a4866b5e50593b9003, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:06.887331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:06.900342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1073/2150] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:06 2022] PUT /identity/v3/projects/6e02c27e6c5a4d318b46bb767599ee8b/users/c920088084d849a4866b5e50593b9003/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:00:06.910514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:06.910807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:06.911084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:06.939399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] MFA Rules not processed for user `c920088084d849a4866b5e50593b9003`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:06.940712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:07.005227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1078/2151] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:07.010406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.031767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:07.077302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0f65b-2194-4891-bc98-a5e27bd40b65 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.082154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.082885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.083174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.083450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.084102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.086056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.113274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1074/2152] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:00:07 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:07.117384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.128533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f453098-2ccc-45d1-823e-c56686956f6f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.185540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.186153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.186431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.186703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.187326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.189264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.210804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1079/2153] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:00:07 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:07.215161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.225288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e67dd2-b1ff-4c07-b714-f4eaeba2bdd3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.230198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.230794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.231074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.231348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.231986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.233638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.244490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1075/2154] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:07.248680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.260359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b2231-8406-4a6f-8853-d8110751bb86 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.265227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.266019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.266304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.266584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] PATH_INFO: `/v3/projects/93c9c37969e94b0ebb6051668e58494e/users/eaa521569497474c814718119d9c3402/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.284373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c9c37969e94b0ebb6051668e58494e, user_id=eaa521569497474c814718119d9c3402, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.288504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.303706 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1080/2155] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:07 2022] PUT /identity/v3/projects/93c9c37969e94b0ebb6051668e58494e/users/eaa521569497474c814718119d9c3402/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:07.308182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.318672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763b4a-6510-44cd-8d68-0c6ffe039f39 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.377463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521c187-5217-4d64-a2c6-d60db7adb013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.378097 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c187-5217-4d64-a2c6-d60db7adb013 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.378376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c187-5217-4d64-a2c6-d60db7adb013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.378648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521c187-5217-4d64-a2c6-d60db7adb013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.379288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521c187-5217-4d64-a2c6-d60db7adb013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.381261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521c187-5217-4d64-a2c6-d60db7adb013 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.394059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1076/2156] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:07.398386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.409236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb622a5-5c8d-4a12-98cb-7e4f77b3b9a4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.414352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.415030 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.415318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.415594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] PATH_INFO: `/v3/projects/93c9c37969e94b0ebb6051668e58494e/users/eaa521569497474c814718119d9c3402/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.417775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c9c37969e94b0ebb6051668e58494e, user_id=eaa521569497474c814718119d9c3402, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.420440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.431788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1081/2157] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:07 2022] PUT /identity/v3/projects/93c9c37969e94b0ebb6051668e58494e/users/eaa521569497474c814718119d9c3402/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:07.443190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.443478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.443805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.472040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] MFA Rules not processed for user `eaa521569497474c814718119d9c3402`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:07.473295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:07.542136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1077/2158] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:07.552614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:07.554328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:07.569311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:07.616711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839befb-8793-4408-b2bb-769961becc9d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:07.621027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987076-f99d-4231-9056-7870d95e86bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:07.621672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987076-f99d-4231-9056-7870d95e86bc service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:07.621947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987076-f99d-4231-9056-7870d95e86bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:07.622212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987076-f99d-4231-9056-7870d95e86bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:07.680092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25987076-f99d-4231-9056-7870d95e86bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:07.681753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25987076-f99d-4231-9056-7870d95e86bc service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:07.690651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1082/2159] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:12.609832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:12.611438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:12.630785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:12.680466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525cf72-2c01-4a96-be19-c560ee4ec214 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:12.685894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:12.686588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:12.686874 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:12.687164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] PATH_INFO: `/v3/users/803a2290376d4333bdec73aa902e73b5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:12.688391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=803a2290376d4333bdec73aa902e73b5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:12.690423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:12.739152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803a2290376d4333bdec73aa902e73b5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:12.747397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803a2290376d4333bdec73aa902e73b5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:12.760507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803a2290376d4333bdec73aa902e73b5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:12.761264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803a2290376d4333bdec73aa902e73b5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:12.768534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803a2290376d4333bdec73aa902e73b5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:12.774541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1078/2160] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:12 2022] DELETE /identity/v3/users/803a2290376d4333bdec73aa902e73b5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:00:12.779213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:12.779696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:12.779834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:12.801166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:12.802370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:12.860241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1083/2161] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:12.870749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:12.872155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:12.928626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436c825-33ec-43f6-a028-f3e930c03a35 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:12.932976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:12.933575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:12.933854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:12.934128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:12.943026 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:12.944582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:12.954695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1079/2162] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:13.228918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:13.230432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:13.242417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a556f4-3aa0-4a95-a201-64357c71fcca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:13.247494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:13.248128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:13.248442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:13.248719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] PATH_INFO: `/v3/projects/127991e9e2a04562b68a481df832c646` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:13.250046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127991e9e2a04562b68a481df832c646)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:13.251943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:13.301918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127991e9e2a04562b68a481df832c646', 'request_id': 'req-b3564b4f-50cc-4b8e-b86c-5725a411dba8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.307597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127991e9e2a04562b68a481df832c646', 'request_id': 'req-b3564b4f-50cc-4b8e-b86c-5725a411dba8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.309456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:13.314192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127991e9e2a04562b68a481df832c646', 'request_id': 'req-b3564b4f-50cc-4b8e-b86c-5725a411dba8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.328383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] The token cache is being invalidate because project 127991e9e2a04562b68a481df832c646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:13.328680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.329341 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.333906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1084/2163] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:13 2022] DELETE /identity/v3/projects/127991e9e2a04562b68a481df832c646 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:00:13.335109 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:13.345858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa183a02-4862-4440-9e43-361bea2091ac service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:13.380648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:13.424514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:13.425366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:13.425707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:13.426054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] PATH_INFO: `/v3/users/7b833bd25f5449f98d8c3bb12a29edae` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:13.427233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b833bd25f5449f98d8c3bb12a29edae)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:13.429250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:13.477583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b833bd25f5449f98d8c3bb12a29edae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.485182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b833bd25f5449f98d8c3bb12a29edae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.496917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b833bd25f5449f98d8c3bb12a29edae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.497628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b833bd25f5449f98d8c3bb12a29edae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.509692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b833bd25f5449f98d8c3bb12a29edae'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.516481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1080/2164] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:13 2022] DELETE /identity/v3/users/7b833bd25f5449f98d8c3bb12a29edae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:13.786372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:13.787823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:13.802499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3564b4f-50cc-4b8e-b86c-5725a411dba8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:13.860454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:13.861090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:13.861368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:13.861659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] PATH_INFO: `/v3/projects/52952206c10c495d9f46af20826bde9f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:13.862749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52952206c10c495d9f46af20826bde9f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:13.864731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:13.911141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52952206c10c495d9f46af20826bde9f', 'request_id': 'req-5d73a199-846f-4e9c-8a43-be3169685260'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.917130 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52952206c10c495d9f46af20826bde9f', 'request_id': 'req-5d73a199-846f-4e9c-8a43-be3169685260'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.923178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52952206c10c495d9f46af20826bde9f', 'request_id': 'req-5d73a199-846f-4e9c-8a43-be3169685260'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.935450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] The token cache is being invalidate because project 52952206c10c495d9f46af20826bde9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:13.935739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.936384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:13.937837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1085/2165] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:13 2022] DELETE /identity/v3/projects/52952206c10c495d9f46af20826bde9f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:13.941850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:13.943222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:13.956019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002e98b-51db-4658-a971-206199878c4b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.015453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.016083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.016395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.016678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] PATH_INFO: `/v3/users/9f91c46ca942428289c018f9ba0bcb14` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.018044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f91c46ca942428289c018f9ba0bcb14)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.019974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.067676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f91c46ca942428289c018f9ba0bcb14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.074860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f91c46ca942428289c018f9ba0bcb14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.076807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:14.091069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f91c46ca942428289c018f9ba0bcb14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.091784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f91c46ca942428289c018f9ba0bcb14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.099249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ff507f04-a5a2-4f4b-9500-9f81d94f214d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f91c46ca942428289c018f9ba0bcb14'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.105964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1081/2166] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:13 2022] DELETE /identity/v3/users/9f91c46ca942428289c018f9ba0bcb14 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:14.380253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.381740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:14.395533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a199-846f-4e9c-8a43-be3169685260 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.464646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.465298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.465578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.465847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] PATH_INFO: `/v3/projects/c78cade25b0044e0a258740b1500ce72` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.467001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78cade25b0044e0a258740b1500ce72)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.468941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.492027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:14.515975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78cade25b0044e0a258740b1500ce72', 'request_id': 'req-69d33221-96bd-49bd-bd14-1afc87227832'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.521744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78cade25b0044e0a258740b1500ce72', 'request_id': 'req-69d33221-96bd-49bd-bd14-1afc87227832'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.528518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78cade25b0044e0a258740b1500ce72', 'request_id': 'req-69d33221-96bd-49bd-bd14-1afc87227832'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.540510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] The token cache is being invalidate because project c78cade25b0044e0a258740b1500ce72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:14.540802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.541414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69d33221-96bd-49bd-bd14-1afc87227832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:14.542883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1086/2167] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:14 2022] DELETE /identity/v3/projects/c78cade25b0044e0a258740b1500ce72 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:14.547972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.548255 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.548551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.568566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:14.569762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:14.630086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1082/2168] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:00:14.637898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.638191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.638469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.659472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:14.660729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:14.669247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1087/2169] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:00:14.673527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.683148 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0673b80-ea6c-4e14-854d-1576f2e502e3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.687658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.688256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.688566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.688841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.689477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:00:14.689778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.691372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.703418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1083/2170] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:00:14.708407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.718523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b603-6c1b-49a3-ba2f-cf5da1a87f07 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.728403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.729020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.729300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.729570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.730221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.732256 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.762015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1088/2171] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:00:14 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:00:14.765813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.778327 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1fc27-edae-4d59-8f80-48a16732d696 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.838335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.838940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.839225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.839499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.840125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.842078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.863404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1084/2172] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:00:14 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:00:14.867790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.878019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25357e-c813-4f72-a215-9acacbbdc73b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.882621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.883155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.883432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.883701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.884701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.886283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.896320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1089/2173] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:14.900674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.910399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88eb4e-3107-475d-b164-e03fdaf54cdc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.914938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f3919-3202-461a-ad53-6e0e751c9157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.916180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f3919-3202-461a-ad53-6e0e751c9157 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.916502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f3919-3202-461a-ad53-6e0e751c9157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.917047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f3919-3202-461a-ad53-6e0e751c9157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.917943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f3919-3202-461a-ad53-6e0e751c9157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.919534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f3919-3202-461a-ad53-6e0e751c9157 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.929799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1085/2174] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:00:14.933825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:14.943709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ba3cc-12c9-4b73-a336-37080f4a0d87 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:14.948527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:14.949178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.949462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.949737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] PATH_INFO: `/v3/projects/d059a60bc4ca4bfc81dee14aaf7cf6ef/users/e3f4a74ab4f4499e8ce29c83cacc3fab/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:14.964857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d059a60bc4ca4bfc81dee14aaf7cf6ef, user_id=e3f4a74ab4f4499e8ce29c83cacc3fab, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:14.967505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:14.980732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1090/2175] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:00:14 2022] PUT /identity/v3/projects/d059a60bc4ca4bfc81dee14aaf7cf6ef/users/e3f4a74ab4f4499e8ce29c83cacc3fab/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:14.992038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:14.992623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:14.992623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:15.020439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] MFA Rules not processed for user `e3f4a74ab4f4499e8ce29c83cacc3fab`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:15.021624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:15.051371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:15.087860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1086/2176] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:15.099411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:15.100610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:15.157745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1558b-45c5-4b17-a77a-4e0ff337a759 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:15.161330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:15.161930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:15.162210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:15.162482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:15.171523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:15.172975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:15.182643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1091/2177] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:19.636368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:19.646735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98286277-35eb-4a1e-98f8-550322ce837e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:19.651099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:19.651773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:19.652081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:19.652383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:19.654283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:19.719258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:19.720745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396f7f6-890e-452d-a572-d1c9dbebad08 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:19.730293 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1087/2178] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:25.177773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:29.725844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=43 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:34.487667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:34.489121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:34.585579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a08186-ad7d-4844-8805-43b3f524fee4 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:34.593907 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:34.595119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:34.595465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:34.595811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] PATH_INFO: `/v3/users/6b8f012a6f3f45ca84781897b5f539e0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:34.597134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8f012a6f3f45ca84781897b5f539e0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:34.600161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:34.695259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8f012a6f3f45ca84781897b5f539e0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:34.709815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8f012a6f3f45ca84781897b5f539e0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:34.717241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8f012a6f3f45ca84781897b5f539e0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:34.726129 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8f012a6f3f45ca84781897b5f539e0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:34.726812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8f012a6f3f45ca84781897b5f539e0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:34.733629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1092/2179] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:34 2022] DELETE /identity/v3/users/6b8f012a6f3f45ca84781897b5f539e0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:34.738225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:34.738713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:34.738804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:34.758980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:34.760207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:34.830679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1088/2180] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:34.841188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:34.842722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:34.902288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06140-3874-4ede-b1f6-c7496e086e7a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:34.906403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:34.907047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:34.907439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:34.907718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:34.919471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:34.922198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:34.936651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1093/2181] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:00:35.240781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:35.242221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:35.255768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb32903-abfd-4560-8f97-edd4eee622cd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:35.267830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:35.267830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:35.267830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:35.268135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] PATH_INFO: `/v3/projects/a207888f7afd4c66839de9f9c5da63c4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:35.269533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a207888f7afd4c66839de9f9c5da63c4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:35.271797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:35.322208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a207888f7afd4c66839de9f9c5da63c4', 'request_id': 'req-ebeaee86-7960-467f-a8ac-289e29c90545'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.328381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a207888f7afd4c66839de9f9c5da63c4', 'request_id': 'req-ebeaee86-7960-467f-a8ac-289e29c90545'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.334458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a207888f7afd4c66839de9f9c5da63c4', 'request_id': 'req-ebeaee86-7960-467f-a8ac-289e29c90545'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.347073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] The token cache is being invalidate because project a207888f7afd4c66839de9f9c5da63c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:35.347323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.347954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ebeaee86-7960-467f-a8ac-289e29c90545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.349472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1089/2182] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:35 2022] DELETE /identity/v3/projects/a207888f7afd4c66839de9f9c5da63c4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:35.719485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:35.721248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:35.734767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7887906-98ef-4c99-8e4e-7c37ea756fa4 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:35.759038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:35.814977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:35.815569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:35.815854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:35.816164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] PATH_INFO: `/v3/users/e3f4a74ab4f4499e8ce29c83cacc3fab` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:35.817405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f4a74ab4f4499e8ce29c83cacc3fab)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:35.819331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:35.888586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f4a74ab4f4499e8ce29c83cacc3fab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.898229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f4a74ab4f4499e8ce29c83cacc3fab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.907822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f4a74ab4f4499e8ce29c83cacc3fab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.916537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f4a74ab4f4499e8ce29c83cacc3fab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.918001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f4a74ab4f4499e8ce29c83cacc3fab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:35.926703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1094/2183] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:00:35 2022] DELETE /identity/v3/users/e3f4a74ab4f4499e8ce29c83cacc3fab => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:35.933235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:35.933965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:35.934059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:35.958879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:00:35.960122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:36.055030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1090/2184] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:00:36.067046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:36.068494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:36.147852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4551fb-8f9a-44ce-8603-e1a14bc34e55 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:36.152340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:36.153211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:36.153496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:36.153769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:36.164241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:36.165857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:36.177065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1095/2185] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:00:36.658615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:00:36.660117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:00:36.672387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945f4ff-798a-44b7-b933-d3708c0f6631 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:00:36.677645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:00:36.678400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:00:36.678889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:00:36.679263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] PATH_INFO: `/v3/projects/d059a60bc4ca4bfc81dee14aaf7cf6ef` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:00:36.680535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d059a60bc4ca4bfc81dee14aaf7cf6ef)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:00:36.683089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:00:36.706926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:36.736736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd059a60bc4ca4bfc81dee14aaf7cf6ef', 'request_id': 'req-e5a5cd91-c735-4c11-9032-b5b581d7437c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:36.743788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd059a60bc4ca4bfc81dee14aaf7cf6ef', 'request_id': 'req-e5a5cd91-c735-4c11-9032-b5b581d7437c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:36.752499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd059a60bc4ca4bfc81dee14aaf7cf6ef', 'request_id': 'req-e5a5cd91-c735-4c11-9032-b5b581d7437c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:36.766607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] The token cache is being invalidate because project d059a60bc4ca4bfc81dee14aaf7cf6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:00:36.766853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:36.767723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:00:36.769247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1091/2186] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:00:36 2022] DELETE /identity/v3/projects/d059a60bc4ca4bfc81dee14aaf7cf6ef => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:00:46.172422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:00:46.763185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:07.424010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:07.425498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:07.439194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791bea2-9417-476b-8974-38c9008a409d service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:07.495468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:07.497658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:07.503181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:07.504051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:07.504548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:07.505036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] PATH_INFO: `/v3/users/c920088084d849a4866b5e50593b9003` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:07.506180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c920088084d849a4866b5e50593b9003)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:07.508241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:07.510697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5cd91-c735-4c11-9032-b5b581d7437c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:07.515504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:07.516141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:07.516446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:07.516718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] PATH_INFO: `/v3/users/c75ef7d06792467b8bf5cf0f40824c2d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:07.518112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75ef7d06792467b8bf5cf0f40824c2d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:07.520100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:07.568178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c920088084d849a4866b5e50593b9003'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.572584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75ef7d06792467b8bf5cf0f40824c2d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.581074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75ef7d06792467b8bf5cf0f40824c2d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.582652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c920088084d849a4866b5e50593b9003'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.591209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c920088084d849a4866b5e50593b9003'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.596319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75ef7d06792467b8bf5cf0f40824c2d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.597081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75ef7d06792467b8bf5cf0f40824c2d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.600623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c920088084d849a4866b5e50593b9003'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.601216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fb077710-1c14-4a65-906b-2630bcecbd46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c920088084d849a4866b5e50593b9003'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.607757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-358fa689-3687-4e06-9150-edc5474c9378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75ef7d06792467b8bf5cf0f40824c2d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:07.609335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1096/2187] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:01:07 2022] DELETE /identity/v3/users/c920088084d849a4866b5e50593b9003 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:07.613580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:07.613872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:07.614155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:07.615615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1092/2188] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:01:07 2022] DELETE /identity/v3/users/c75ef7d06792467b8bf5cf0f40824c2d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:07.619675 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:07.620159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:07.620470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:07.633833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:07.635057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:07.642136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:07.643669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:07.704388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1097/2189] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:01:07.715351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1093/2190] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:07.715689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:07.716005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:07.721170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:07.723045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:07.738419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:07.778503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5651fb-d49f-49ae-8aa1-7c5137f6ff6f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:07.782250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:07.782863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:07.783145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:07.783480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:07.786390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d59951-3e30-4937-a3d7-5883ad82ad71 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:07.790039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:07.790646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:07.790918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:07.791194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:07.794283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:07.795957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:07.803689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:07.805167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:07.807593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1098/2191] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:07.816764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1094/2192] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:01:08.167503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.169780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.188238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b01c9-36d6-49b5-9bc7-d12bf32735a9 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.202769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.203011 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.203869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.204152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.204451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] PATH_INFO: `/v3/projects/6e02c27e6c5a4d318b46bb767599ee8b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.205184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.205637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e02c27e6c5a4d318b46bb767599ee8b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.207531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.219855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f74f31-039f-4d2f-8922-1449b3b69448 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.226252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.226892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.227172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.227446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] PATH_INFO: `/v3/projects/c32a7fa3267240989a0384ea463dd212` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.228637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32a7fa3267240989a0384ea463dd212)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.231895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.242995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:08.266693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e02c27e6c5a4d318b46bb767599ee8b', 'request_id': 'req-97782d27-6d17-4857-a088-8bd2aae3b45e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.272984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e02c27e6c5a4d318b46bb767599ee8b', 'request_id': 'req-97782d27-6d17-4857-a088-8bd2aae3b45e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.279732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e02c27e6c5a4d318b46bb767599ee8b', 'request_id': 'req-97782d27-6d17-4857-a088-8bd2aae3b45e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.284047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32a7fa3267240989a0384ea463dd212', 'request_id': 'req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.290341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32a7fa3267240989a0384ea463dd212', 'request_id': 'req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.293070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] The token cache is being invalidate because project 6e02c27e6c5a4d318b46bb767599ee8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:01:08.293654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.294439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.296046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1099/2193] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:01:08 2022] DELETE /identity/v3/projects/6e02c27e6c5a4d318b46bb767599ee8b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:08.299823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.300012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32a7fa3267240989a0384ea463dd212', 'request_id': 'req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.301157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.313089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] The token cache is being invalidate because project c32a7fa3267240989a0384ea463dd212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:01:08.313383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.314035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f6717bd-e376-4fb6-846b-b16ed71f8ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.315519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1095/2194] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:01:08 2022] DELETE /identity/v3/projects/c32a7fa3267240989a0384ea463dd212 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:08.320143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c224e-86f7-451f-acb0-dfad5a543003 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.320514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c224e-86f7-451f-acb0-dfad5a543003 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.320821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c224e-86f7-451f-acb0-dfad5a543003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.340210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ed7c224e-86f7-451f-acb0-dfad5a543003 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:08.341439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ed7c224e-86f7-451f-acb0-dfad5a543003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.368134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97782d27-6d17-4857-a088-8bd2aae3b45e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.407730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1096/2195] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:08.415640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.415995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.416531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.427232 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:08.440392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:08.441019 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.441606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.441751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.442173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.442524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] PATH_INFO: `/v3/users/eaa521569497474c814718119d9c3402` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.443768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa521569497474c814718119d9c3402)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.445801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.450474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1097/2196] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:01:08.454546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.463783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fc6a6-db73-42d0-952d-f2b2f36f7b1c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.468896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.469584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.469924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.470261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.471004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:01:08.471374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.473144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.488311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1098/2197] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:01:08.492206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.503690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f924970-691f-4e39-a6de-04359c2eb398 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.506657 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa521569497474c814718119d9c3402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.509849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.510584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.510868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.511143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.512831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.514246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.522229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa521569497474c814718119d9c3402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.530617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa521569497474c814718119d9c3402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.539658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa521569497474c814718119d9c3402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.540473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa521569497474c814718119d9c3402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.559301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1099/2198] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:01:08 2022] POST /identity/v3/projects => generated 363 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:01:08.562031 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1100/2199] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:01:08 2022] DELETE /identity/v3/users/eaa521569497474c814718119d9c3402 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:08.563585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.565312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.628578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:08.632990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e161e1-e58a-4537-bb31-0a6322e539f4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.693838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.694572 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.694855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.695120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.695762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.697695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.724628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1100/2200] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:01:08 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:01:08.729053 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.739610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c208161-3d01-48be-b5cf-0bdf88de7289 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.744147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.744780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.745058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.745326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.746213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.747971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.759833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1101/2201] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:01:08.762596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.773077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e698e-21c1-4d0d-baaa-dc094ce33ce2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.778312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.778910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.779186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.779457 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.780092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.781729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.791714 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1101/2202] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:08.796588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.807627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69277b32-6010-4876-bd6a-726136c4c2ab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.812616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bdef9-8b16-4879-8627-d225091f8ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.813267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bdef9-8b16-4879-8627-d225091f8ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.813544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bdef9-8b16-4879-8627-d225091f8ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.813814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bdef9-8b16-4879-8627-d225091f8ee2 admin admin] PATH_INFO: `/v3/projects/9a8274f4532c4076b47d3bc93e03270b/users/2f7711aea6094976826571bf92746552/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.830780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bdef9-8b16-4879-8627-d225091f8ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8274f4532c4076b47d3bc93e03270b, user_id=2f7711aea6094976826571bf92746552, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.833529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bdef9-8b16-4879-8627-d225091f8ee2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.845784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.847937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.856456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1102/2203] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:01:08 2022] PUT /identity/v3/projects/9a8274f4532c4076b47d3bc93e03270b/users/2f7711aea6094976826571bf92746552/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:08.860777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.861077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.861357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.872143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e104a3-d2b4-40d6-9543-7bd43d75a005 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:08.893854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] MFA Rules not processed for user `2f7711aea6094976826571bf92746552`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:08.895029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.931045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:08.931672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:08.931950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:08.932222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] PATH_INFO: `/v3/projects/93c9c37969e94b0ebb6051668e58494e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:08.934181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c9c37969e94b0ebb6051668e58494e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:08.936892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:08.962067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1103/2204] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:01:08.974404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:08.975842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:08.985140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c9c37969e94b0ebb6051668e58494e', 'request_id': 'req-adc57f68-00a4-4ad2-a444-06559ed48cc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.991012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c9c37969e94b0ebb6051668e58494e', 'request_id': 'req-adc57f68-00a4-4ad2-a444-06559ed48cc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:08.997736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c9c37969e94b0ebb6051668e58494e', 'request_id': 'req-adc57f68-00a4-4ad2-a444-06559ed48cc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:09.008180 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:09.010155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] The token cache is being invalidate because project 93c9c37969e94b0ebb6051668e58494e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:01:09.010450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:09.011112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-adc57f68-00a4-4ad2-a444-06559ed48cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:09.013222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1102/2205] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:01:08 2022] DELETE /identity/v3/projects/93c9c37969e94b0ebb6051668e58494e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:09.017736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a460e4-32e1-4c58-ba43-08eca72c0ab2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.018954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a460e4-32e1-4c58-ba43-08eca72c0ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.018954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a460e4-32e1-4c58-ba43-08eca72c0ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.037966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-70a460e4-32e1-4c58-ba43-08eca72c0ab2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:09.039161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-70a460e4-32e1-4c58-ba43-08eca72c0ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:09.044176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e2f03-cce1-4756-a52e-77dcf294ef5b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.048212 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:09.094265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.094885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.095175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.095478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.097318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:09.113772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1103/2206] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:01:09.123178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.123469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.123750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.144205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:09.145617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:09.153742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1104/2207] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:09.157939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.159725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.161228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.166711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fdd7e-3bb4-4f61-8c96-21da173500ba None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.170758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1104/2208] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:01:09.171975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.172613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.172886 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.173155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.173788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:01:09.174085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.175667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.188018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1105/2209] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:01:09.192441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.203343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f191d-af9e-4838-a5aa-0a4cd08c3915 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.207924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.208551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.208834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.209111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.209747 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.211619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.240303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1105/2210] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:01:09.243752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.254623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649797f9-6cff-4b34-9b4d-516e2c1a1bfb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.316735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.317352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.317627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.317900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.318529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.320922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.344417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1106/2211] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/users => generated 305 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:01:09.348696 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.359565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0457e1-ae94-47e8-9c3b-91a01bacf65a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.364576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.365181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.365469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.365743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.366705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.368189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.380028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1106/2212] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:09.384637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.396192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6b069-7f55-4f87-96f7-df7157627df6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.403369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb921-16cc-4913-ab26-5074f19ed165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.403981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb921-16cc-4913-ab26-5074f19ed165 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.404318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb921-16cc-4913-ab26-5074f19ed165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.404613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb921-16cc-4913-ab26-5074f19ed165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.405263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb921-16cc-4913-ab26-5074f19ed165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.406939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb921-16cc-4913-ab26-5074f19ed165 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.432454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1107/2213] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:09.435147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.448490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a9e6e-808f-4ff4-a231-53cf58e3755a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.455055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.455723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.456009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.456303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] PATH_INFO: `/v3/projects/ed4f87e571174d48b424dd9c5dfdfe3b/users/6f18721ee5604ff8a96650ad9ba5f0f9/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.471850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4f87e571174d48b424dd9c5dfdfe3b, user_id=6f18721ee5604ff8a96650ad9ba5f0f9, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.474525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.488647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1107/2214] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:01:09 2022] PUT /identity/v3/projects/ed4f87e571174d48b424dd9c5dfdfe3b/users/6f18721ee5604ff8a96650ad9ba5f0f9/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:09.500182 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed37856-4956-4130-848a-def06f8cab0c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.500500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed37856-4956-4130-848a-def06f8cab0c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.500814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed37856-4956-4130-848a-def06f8cab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.531356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bed37856-4956-4130-848a-def06f8cab0c None None] MFA Rules not processed for user `6f18721ee5604ff8a96650ad9ba5f0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:09.532596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bed37856-4956-4130-848a-def06f8cab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:09.598398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1108/2215] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:09.602945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.649879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:09.671969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2725f6-d6c9-460e-b444-c4b2fbd399d4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.676882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.677500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.677784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.678060 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.678738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.680816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.710232 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1108/2216] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:01:09.714139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed37856-4956-4130-848a-def06f8cab0c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.720068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:09.733340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed37856-4956-4130-848a-def06f8cab0c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.787743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.788390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.788674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.788945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.789569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.791490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.813835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1109/2217] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:01:09 2022] POST /identity/v3/users => generated 310 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:01:09.817878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.828600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59500edd-b543-4203-bd68-b0db16c5bbed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.833141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.833732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.834013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.834286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.834915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.836520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.847243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1109/2218] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:09.851345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.861911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c2ebb-72a0-45b1-be76-f47f72ad408c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.866550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.867193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.867474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.867751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] PATH_INFO: `/v3/projects/f9a44eb399da4299a39b07387880a33c/users/9b2c3b412ad246fbbdd2291fb1937c38/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.882760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a44eb399da4299a39b07387880a33c, user_id=9b2c3b412ad246fbbdd2291fb1937c38, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.885437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.898129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1110/2219] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:01:09 2022] PUT /identity/v3/projects/f9a44eb399da4299a39b07387880a33c/users/9b2c3b412ad246fbbdd2291fb1937c38/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:09.902214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.912043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781009a-e027-4a9b-80ee-3cdb452bfca4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:09.970362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df0fd6-84a2-40cd-8cb2-18b22ac0d9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:09.970975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df0fd6-84a2-40cd-8cb2-18b22ac0d9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:09.971259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df0fd6-84a2-40cd-8cb2-18b22ac0d9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:09.971535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df0fd6-84a2-40cd-8cb2-18b22ac0d9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:09.972170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df0fd6-84a2-40cd-8cb2-18b22ac0d9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:09.973800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df0fd6-84a2-40cd-8cb2-18b22ac0d9d1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:09.984515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1110/2220] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:09.988492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:09.999618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d2f55-fa12-4807-a33f-b6b3e306b7b6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:10.004225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:10.004901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:10.005178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:10.005444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] PATH_INFO: `/v3/projects/f9a44eb399da4299a39b07387880a33c/users/9b2c3b412ad246fbbdd2291fb1937c38/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:10.007320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a44eb399da4299a39b07387880a33c, user_id=9b2c3b412ad246fbbdd2291fb1937c38, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:10.009971 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:10.020537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1111/2221] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:01:09 2022] PUT /identity/v3/projects/f9a44eb399da4299a39b07387880a33c/users/9b2c3b412ad246fbbdd2291fb1937c38/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:10.031507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:10.031802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:10.032083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:10.060304 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] MFA Rules not processed for user `9b2c3b412ad246fbbdd2291fb1937c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:10.061534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:10.134325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1111/2222] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:10 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:10.145839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:10.147589 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:10.206058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a7f3-90de-4523-bc2c-57fa41d47098 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:10.209681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:10.210276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:10.210585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:10.210855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:10.220602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:10.222025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:10.232071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1112/2223] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:10 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:10.386164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:10.396485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7df63-da59-4e45-afd6-73136b1e1bdb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:10.399982 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f9431-a5cf-4262-b286-d10e1b4b784d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:10.400631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f9431-a5cf-4262-b286-d10e1b4b784d service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:10.400909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f9431-a5cf-4262-b286-d10e1b4b784d service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:10.401177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f9431-a5cf-4262-b286-d10e1b4b784d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:10.434080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:10.459297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f9431-a5cf-4262-b286-d10e1b4b784d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:10.460698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f9431-a5cf-4262-b286-d10e1b4b784d service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:10.470135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1112/2224] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:10 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:19.892950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:19.969025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:19.973773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725ee5d-7096-4cec-8f13-5528aa47850c service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:19.983929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:19.984621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:19.984924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:19.985333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] PATH_INFO: `/v3/users/6f18721ee5604ff8a96650ad9ba5f0f9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:19.986403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f18721ee5604ff8a96650ad9ba5f0f9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:19.988831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:20.058918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f18721ee5604ff8a96650ad9ba5f0f9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.068006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f18721ee5604ff8a96650ad9ba5f0f9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.102005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f18721ee5604ff8a96650ad9ba5f0f9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.102987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f18721ee5604ff8a96650ad9ba5f0f9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.114868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f18721ee5604ff8a96650ad9ba5f0f9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.123733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1113/2225] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:01:19 2022] DELETE /identity/v3/users/6f18721ee5604ff8a96650ad9ba5f0f9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:01:20.128224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d891c-4ef1-4876-9c31-e571bffc84ce None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:20.128746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d891c-4ef1-4876-9c31-e571bffc84ce None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:20.128959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d891c-4ef1-4876-9c31-e571bffc84ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:20.154905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4a0d891c-4ef1-4876-9c31-e571bffc84ce None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:20.156209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a0d891c-4ef1-4876-9c31-e571bffc84ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:20.218717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1113/2226] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:01:20.228557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:20.230077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:20.289336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1251b36-9764-4699-8b54-a3aafdd3285b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:20.295415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:20.296038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:20.296351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:20.296633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:20.306023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:20.307454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:20.317732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1114/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:20.442584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:20.443067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:20.443434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:20.462558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] MFA Rules not processed for user `e2ef1277bf944722aa33077985fb4734`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:20.463874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:20.526386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1114/2228] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:20.596339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:20.597928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:20.611822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb38de3-d21c-4aef-8b1c-fceb606ba1cf service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:20.616819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:20.617586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:20.617888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:20.618230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] PATH_INFO: `/v3/projects/ed4f87e571174d48b424dd9c5dfdfe3b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:20.619925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4f87e571174d48b424dd9c5dfdfe3b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:20.622262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:20.675420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4f87e571174d48b424dd9c5dfdfe3b', 'request_id': 'req-03b5cb83-fa56-4db9-8201-d8e805db44e6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.681335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4f87e571174d48b424dd9c5dfdfe3b', 'request_id': 'req-03b5cb83-fa56-4db9-8201-d8e805db44e6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.688627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4f87e571174d48b424dd9c5dfdfe3b', 'request_id': 'req-03b5cb83-fa56-4db9-8201-d8e805db44e6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.702102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] The token cache is being invalidate because project ed4f87e571174d48b424dd9c5dfdfe3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:01:20.702420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.703044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.704593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1115/2229] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:01:20 2022] DELETE /identity/v3/projects/ed4f87e571174d48b424dd9c5dfdfe3b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:20.708640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:20.719501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980aedb-208e-4208-b11a-0c8daf90b0db None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:20.779569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:20.780211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:20.780527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:20.780804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] PATH_INFO: `/v3/users/9b2c3b412ad246fbbdd2291fb1937c38` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:20.781929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b2c3b412ad246fbbdd2291fb1937c38)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:20.783857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:20.806411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:20.832299 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b2c3b412ad246fbbdd2291fb1937c38'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.845536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b2c3b412ad246fbbdd2291fb1937c38'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.852492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b2c3b412ad246fbbdd2291fb1937c38'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.861379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b2c3b412ad246fbbdd2291fb1937c38'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.862106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f2edffaf-d23d-41e9-9197-7cc9e8d1cadc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b2c3b412ad246fbbdd2291fb1937c38'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:20.868151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1115/2230] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:01:20 2022] DELETE /identity/v3/users/9b2c3b412ad246fbbdd2291fb1937c38 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:21.227248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.229013 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:21.239530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:21.265404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5cb83-fa56-4db9-8201-d8e805db44e6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.352301 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.353028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.353381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.353760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] PATH_INFO: `/v3/projects/f9a44eb399da4299a39b07387880a33c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.356415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a44eb399da4299a39b07387880a33c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.358704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.415155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a44eb399da4299a39b07387880a33c', 'request_id': 'req-0686e4bb-7aec-417a-b3db-a67ca19dcb03'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:21.422304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a44eb399da4299a39b07387880a33c', 'request_id': 'req-0686e4bb-7aec-417a-b3db-a67ca19dcb03'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:21.429799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a44eb399da4299a39b07387880a33c', 'request_id': 'req-0686e4bb-7aec-417a-b3db-a67ca19dcb03'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:21.443048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] The token cache is being invalidate because project f9a44eb399da4299a39b07387880a33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:01:21.443342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:21.444086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0686e4bb-7aec-417a-b3db-a67ca19dcb03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:21.446310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1116/2231] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:01:21 2022] DELETE /identity/v3/projects/f9a44eb399da4299a39b07387880a33c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:21.452107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.452615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.452859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.500083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:21.501577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:21.600844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1116/2232] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:21.610609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.611010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.611298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.631626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:21.632865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:21.641902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1117/2233] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:21.646516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.658532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d409c-cc61-480c-a343-58c7d1f81fcb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.663714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.664259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.664566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.664841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.665489 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:01:21.665795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.667396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.679964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1117/2234] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:01:21.684456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.693528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c91c3c-9df8-4e6b-b0b5-1e7b77e6b45c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.698160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.698749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.699545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.699833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.700486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.702375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.733123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1118/2235] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:01:21 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:01:21.737262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.748179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3379f-d68a-4a1f-9e44-a1b1dd19fed9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.816124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.816771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.817083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.817360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.818002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.819900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.841119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1118/2236] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:01:21 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:01:21.845321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.856014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4369b4d-71bb-44a4-be1b-cc7a3f01754a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.860694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.861283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.861564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.861835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.862471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.864076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.874622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1119/2237] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:21.878469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.888589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011a087-6f43-4369-a310-ce60f467bd0c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.893472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58077a1a-d005-47ea-bbe4-0ea1803f67b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.894076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58077a1a-d005-47ea-bbe4-0ea1803f67b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.894356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58077a1a-d005-47ea-bbe4-0ea1803f67b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.894627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58077a1a-d005-47ea-bbe4-0ea1803f67b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.895283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58077a1a-d005-47ea-bbe4-0ea1803f67b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.896896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58077a1a-d005-47ea-bbe4-0ea1803f67b7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.907695 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1119/2238] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:21.911841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:21.922413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8431f14-f1f8-4817-beb3-cb0e0f398e20 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:21.926989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:21.927641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.927924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.928200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] PATH_INFO: `/v3/projects/9b5e5be6e7a543c1b139507496c2a01d/users/10bdf28e11764cf2804c92fd70470472/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.944812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5e5be6e7a543c1b139507496c2a01d, user_id=10bdf28e11764cf2804c92fd70470472, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:21.947494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:21.961380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1120/2239] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:01:21 2022] PUT /identity/v3/projects/9b5e5be6e7a543c1b139507496c2a01d/users/10bdf28e11764cf2804c92fd70470472/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:21.972325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:21.972501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:21.972769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:21.982843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:22.005435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] MFA Rules not processed for user `10bdf28e11764cf2804c92fd70470472`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:22.006669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:22.075966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1120/2240] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:22.088828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:22.091348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:22.132397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:22.161515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d775cf-9057-4a1c-8823-b87b97e98423 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:22.165331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:22.165965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:22.166246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:22.166521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:22.174768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:22.176198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:22.185439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1121/2241] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:32.072612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=30 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:32.180870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=13 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:40.829113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:40.830574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:40.910354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5272cc-474f-472e-8ada-5b0ca83cf2ca None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:40.915595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:40.916391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:40.916680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:40.917401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:40.919439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:40.932360 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:40.934218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a50325-a86e-4378-8032-220599853ea8 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:40.946527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1121/2242] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:50.941162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=32 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:51.432757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:51.504580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d072b7-34df-4bb3-93f0-cebfa31b4856 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:51.509322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:51.509966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:51.510247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:51.510517 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] PATH_INFO: `/v3/users/10bdf28e11764cf2804c92fd70470472` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:51.511667 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bdf28e11764cf2804c92fd70470472)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:51.514273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:51.568113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bdf28e11764cf2804c92fd70470472'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:51.576206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bdf28e11764cf2804c92fd70470472'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:51.589607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bdf28e11764cf2804c92fd70470472'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:51.590346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bdf28e11764cf2804c92fd70470472'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:51.598499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bdf28e11764cf2804c92fd70470472'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:51.605156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1122/2243] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:01:51 2022] DELETE /identity/v3/users/10bdf28e11764cf2804c92fd70470472 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:51.611201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:51.611809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:51.612155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:51.632220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:51.635097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:51.697228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1122/2244] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:51.707312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:51.709146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:51.776193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea541e-37c8-4833-be53-829aa09b5af9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:51.780031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5af52-befc-4bfd-8b2f-12d75c6fa376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:51.780683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5af52-befc-4bfd-8b2f-12d75c6fa376 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:51.780963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5af52-befc-4bfd-8b2f-12d75c6fa376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:51.781369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5af52-befc-4bfd-8b2f-12d75c6fa376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:51.791251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5af52-befc-4bfd-8b2f-12d75c6fa376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:51.792746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5af52-befc-4bfd-8b2f-12d75c6fa376 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:51.805924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1123/2245] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:01:52.265955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.268359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:52.282690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f159c-5749-4730-8bb4-21fb692c0317 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.289223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.289882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.291434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.291727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] PATH_INFO: `/v3/projects/9b5e5be6e7a543c1b139507496c2a01d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.293390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5e5be6e7a543c1b139507496c2a01d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.296427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.358334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5e5be6e7a543c1b139507496c2a01d', 'request_id': 'req-e1849f93-e37a-4ddb-a47f-4219eb4e3752'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:52.364978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5e5be6e7a543c1b139507496c2a01d', 'request_id': 'req-e1849f93-e37a-4ddb-a47f-4219eb4e3752'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:52.375476 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5e5be6e7a543c1b139507496c2a01d', 'request_id': 'req-e1849f93-e37a-4ddb-a47f-4219eb4e3752'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:52.395750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] The token cache is being invalidate because project 9b5e5be6e7a543c1b139507496c2a01d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:01:52.395750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:52.396485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e1849f93-e37a-4ddb-a47f-4219eb4e3752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:01:52.398683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1123/2246] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:01:52 2022] DELETE /identity/v3/projects/9b5e5be6e7a543c1b139507496c2a01d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:52.405265 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1772089-99f4-44ec-b968-aa092586558d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.405777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1772089-99f4-44ec-b968-aa092586558d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.406225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1772089-99f4-44ec-b968-aa092586558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.429990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c1772089-99f4-44ec-b968-aa092586558d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:52.431226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1772089-99f4-44ec-b968-aa092586558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:52.453367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:52.509380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1124/2247] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:01:52.520187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.521122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.522751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.547887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:52.549248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:52.559155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1124/2248] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:52.563600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1772089-99f4-44ec-b968-aa092586558d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.574132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1772089-99f4-44ec-b968-aa092586558d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.583254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.584662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.584885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.585154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.585831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:01:52.586139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.587761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.601444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1125/2249] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:01:52.605871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.615837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b0a18-fc94-407d-ae19-93f401aa802f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.621002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.621823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.622505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.622945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.623900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.626073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.661704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1125/2250] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:01:52 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:01:52.662895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.675967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaeb7be-27ce-4c84-8a94-e700fb15850d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.740611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.741227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.741573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.741864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.742505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.744486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.769878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1126/2251] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:01:52 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:01:52.771205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.785336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1791d68-ade7-4662-9623-a8c9e655e1af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.790166 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.790785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.791071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.791350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.791999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.793645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.807342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1126/2252] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:01:52.811791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.822569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173c94f-d5ec-4e32-a12c-c1f442ede3e0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.827593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55418-a08b-4543-98aa-1160b9d5ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.828205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55418-a08b-4543-98aa-1160b9d5ac85 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.828522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55418-a08b-4543-98aa-1160b9d5ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.828912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55418-a08b-4543-98aa-1160b9d5ac85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.829629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b55418-a08b-4543-98aa-1160b9d5ac85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.831562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b55418-a08b-4543-98aa-1160b9d5ac85 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.848305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1127/2253] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:01:52.850390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:52.862009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a32b9-7d15-4077-9b3d-75fe214071f3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:52.866756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:52.867440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.867773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.868069 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] PATH_INFO: `/v3/projects/00551102e4a845ea95e15fa613dfbf59/users/5043da49656d48f29de72f84470739df/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.885212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] RBAC: Authorizing `identity:create_grant(project_id=00551102e4a845ea95e15fa613dfbf59, user_id=5043da49656d48f29de72f84470739df, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:52.888077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:52.905239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1127/2254] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:01:52 2022] PUT /identity/v3/projects/00551102e4a845ea95e15fa613dfbf59/users/5043da49656d48f29de72f84470739df/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:01:52.915812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:52.916168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:52.916552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:52.948495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] MFA Rules not processed for user `5043da49656d48f29de72f84470739df`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:01:52.949883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:53.025321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1128/2255] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:01:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:01:53.036680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:01:53.038692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:01:53.063311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:01:53.108836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596f2dd-c25f-4da4-a94e-f21e9f2f638a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:01:53.112745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015ad2c-6e77-4228-8221-5a7ccfd8a8c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:01:53.113444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015ad2c-6e77-4228-8221-5a7ccfd8a8c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:01:53.113826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015ad2c-6e77-4228-8221-5a7ccfd8a8c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:01:53.114115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015ad2c-6e77-4228-8221-5a7ccfd8a8c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:01:53.123748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015ad2c-6e77-4228-8221-5a7ccfd8a8c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:01:53.125336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015ad2c-6e77-4228-8221-5a7ccfd8a8c0 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:01:53.135066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1128/2256] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:01:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:02:03.021968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:03.131196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=17 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:03.248931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:03.309990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a408e-f808-4eb6-a3a1-b5b650e728e9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:03.314682 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:03.315297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:03.315573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:03.315840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] PATH_INFO: `/v3/users/5043da49656d48f29de72f84470739df` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:03.316952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=5043da49656d48f29de72f84470739df)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:03.318853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:03.366088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5043da49656d48f29de72f84470739df'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.373404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5043da49656d48f29de72f84470739df'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.385374 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5043da49656d48f29de72f84470739df'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.386021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5043da49656d48f29de72f84470739df'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.393405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5043da49656d48f29de72f84470739df'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.399269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1129/2257] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:02:03 2022] DELETE /identity/v3/users/5043da49656d48f29de72f84470739df => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:02:03.403911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:03.404201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:03.404511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:03.424096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:03.425320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:03.483114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1129/2258] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:03.492865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:03.494329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:03.552142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca34013-25d0-429a-b094-3392deeb98ee admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:03.555850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9774be-5319-468f-b53e-dc3a4c537f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:03.556484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9774be-5319-468f-b53e-dc3a4c537f74 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:03.556760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9774be-5319-468f-b53e-dc3a4c537f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:03.557027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9774be-5319-468f-b53e-dc3a4c537f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:03.566933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9774be-5319-468f-b53e-dc3a4c537f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:03.568381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9774be-5319-468f-b53e-dc3a4c537f74 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:03.578569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1130/2259] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:02:03.848803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:03.850324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:03.863195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aecb3-9c07-471b-a8c5-1cda1ab19e71 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:03.867705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:03.868369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:03.868652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:03.868930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] PATH_INFO: `/v3/projects/00551102e4a845ea95e15fa613dfbf59` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:03.870070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00551102e4a845ea95e15fa613dfbf59)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:03.872670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:03.921208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00551102e4a845ea95e15fa613dfbf59', 'request_id': 'req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.926999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00551102e4a845ea95e15fa613dfbf59', 'request_id': 'req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.933558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00551102e4a845ea95e15fa613dfbf59', 'request_id': 'req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.947898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] The token cache is being invalidate because project 00551102e4a845ea95e15fa613dfbf59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:02:03.948185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.948843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d2179d12-9c16-49f6-b2ee-cb1ee6cffc50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:03.950532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1130/2260] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:02:03 2022] DELETE /identity/v3/projects/00551102e4a845ea95e15fa613dfbf59 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:02:03.955485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:03.955776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:03.956048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:03.977116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:03.978337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:03.995510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:04.039403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1131/2261] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:04.048226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.048549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.048827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.068846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:04.070045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:04.078424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1131/2262] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:04.082556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.092525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195048bb-5bb9-4b3d-a5e6-36b91d967c57 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.097237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.097837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.098484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.098484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.099045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:02:04.099347 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.100982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.112749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1132/2263] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:02:04.117052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.126672 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4209ef-791f-4809-8ffa-6d1c1c41384a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.131278 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.131888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.132145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.132451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.133071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.134954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.162156 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1132/2264] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:02:04 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:02:04.166378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.176776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e05ff7-fcd5-4e16-aa89-6b8acd91a47b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.234491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.235126 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.235441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.235748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.236442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.238390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.259557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1133/2265] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:02:04 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:02:04.264188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.274649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bbc99-0041-4292-b6cb-7b5c7cc0b59f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.279342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.279933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.280215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.280515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.281153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.282731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.292459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1133/2266] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:02:04.296295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.306245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9af5e8-b822-4ec7-83e4-9be99e43b170 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.311900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff043b7-d2c4-448c-b3b7-8022331c106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.312547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff043b7-d2c4-448c-b3b7-8022331c106f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.312868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff043b7-d2c4-448c-b3b7-8022331c106f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.313178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff043b7-d2c4-448c-b3b7-8022331c106f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.313851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff043b7-d2c4-448c-b3b7-8022331c106f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.315605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff043b7-d2c4-448c-b3b7-8022331c106f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.327072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1134/2267] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:02:04.331055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.341301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89d089-664c-4b50-bde3-185205d16129 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.345912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.346553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.346837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.347111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] PATH_INFO: `/v3/projects/a6537f34327448c2b6acd5774dffc8c9/users/381e3962c4e24a599e720303e9be17e2/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.361902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6537f34327448c2b6acd5774dffc8c9, user_id=381e3962c4e24a599e720303e9be17e2, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.364569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.378145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1134/2268] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:02:04 2022] PUT /identity/v3/projects/a6537f34327448c2b6acd5774dffc8c9/users/381e3962c4e24a599e720303e9be17e2/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:02:04.383031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f0248-5220-4a64-af93-c2b7e64ebf4d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.383384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f0248-5220-4a64-af93-c2b7e64ebf4d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.383715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f0248-5220-4a64-af93-c2b7e64ebf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.402920 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-212f0248-5220-4a64-af93-c2b7e64ebf4d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:04.404307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-212f0248-5220-4a64-af93-c2b7e64ebf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:04.461543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1135/2269] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:04.471140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:04.472884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:04.495033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:04.531087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e3691-ade9-442f-99b3-6d6504e9c223 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:04.534698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:04.535301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:04.535585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:04.535860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:04.545179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:04.546607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:04.555827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1135/2270] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:02:08.646275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:08.646737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:08.646898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:08.677768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] MFA Rules not processed for user `381e3962c4e24a599e720303e9be17e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:08.679019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:08.698526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:08.748548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1136/2271] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:08.753096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:08.763893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc5c8-68ab-4f69-b0a4-fc8963b12a54 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:08.768768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:08.769365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:08.769637 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:08.769911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:08.770540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:08.772446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:08.801296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1136/2272] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:02:08 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:02:08.805482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:08.823334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183390d-0d8a-467f-b01a-57f3dd65979b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:08.893758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:08.894553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:08.894881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:08.895187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:08.895852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:08.897856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:08.921481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1137/2273] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:02:08 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:02:08.925570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:08.942961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336baa-a4a5-4762-8b01-7289d5fa13a1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:08.948094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:08.948740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:08.949026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:08.949298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:08.950796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:08.952513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:08.964037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1137/2274] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:02:08.968594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:08.979277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc15a3a-c83f-49a9-a918-f6af751b67e4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:08.984736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:08.985379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:08.985696 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:08.986023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] PATH_INFO: `/v3/projects/a6ba2d3a042b4248af4e59f196584800/users/482042dc94d44d20b44c62c475236e08/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:09.002150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ba2d3a042b4248af4e59f196584800, user_id=482042dc94d44d20b44c62c475236e08, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:09.004866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:09.019486 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1138/2275] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:02:08 2022] PUT /identity/v3/projects/a6ba2d3a042b4248af4e59f196584800/users/482042dc94d44d20b44c62c475236e08/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:02:09.023444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:09.034248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727c05d-6b2a-4a56-a106-82924459b922 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:09.097548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab08d5a-2d1b-47d8-a402-e135212fd28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:09.098743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08d5a-2d1b-47d8-a402-e135212fd28c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:09.098743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08d5a-2d1b-47d8-a402-e135212fd28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:09.098922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08d5a-2d1b-47d8-a402-e135212fd28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:09.099412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab08d5a-2d1b-47d8-a402-e135212fd28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:09.101058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab08d5a-2d1b-47d8-a402-e135212fd28c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:09.112508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1138/2276] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:02:09.116724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:09.127194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f9827-308a-43ea-9c64-98eb8ff77112 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:09.131914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:09.132657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:09.132981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:09.133278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] PATH_INFO: `/v3/projects/a6ba2d3a042b4248af4e59f196584800/users/482042dc94d44d20b44c62c475236e08/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:09.135193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ba2d3a042b4248af4e59f196584800, user_id=482042dc94d44d20b44c62c475236e08, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:09.138410 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:09.150529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1139/2277] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:02:09 2022] PUT /identity/v3/projects/a6ba2d3a042b4248af4e59f196584800/users/482042dc94d44d20b44c62c475236e08/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:02:13.521450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6f1b3-5d89-4da4-84c7-1acd8fe59fa1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:13.521915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6f1b3-5d89-4da4-84c7-1acd8fe59fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:13.522009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6f1b3-5d89-4da4-84c7-1acd8fe59fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:13.556075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12d6f1b3-5d89-4da4-84c7-1acd8fe59fa1 None None] MFA Rules not processed for user `482042dc94d44d20b44c62c475236e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:13.557356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12d6f1b3-5d89-4da4-84c7-1acd8fe59fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:13.640461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1139/2278] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:13.662018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:13.663846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:13.742777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd69ef-717c-44e9-8b44-cca8f5034495 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:13.748730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:13.749364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:13.749645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:13.749919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:13.756394 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:13.814901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:13.816561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:13.832636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1140/2279] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:02:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:02:14.884541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74509b2-ce44-4b3f-9169-91b2b6b40d94 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:14.885039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74509b2-ce44-4b3f-9169-91b2b6b40d94 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:14.885153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74509b2-ce44-4b3f-9169-91b2b6b40d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:14.905789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a74509b2-ce44-4b3f-9169-91b2b6b40d94 None None] MFA Rules not processed for user `381e3962c4e24a599e720303e9be17e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:14.907043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a74509b2-ce44-4b3f-9169-91b2b6b40d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:14.914205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1140/2280] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:02:15.771828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:15.773418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:15.848654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef74a0-b9c3-42d3-abea-6dda33250e23 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:15.853297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:15.853875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:15.854156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:15.854431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] PATH_INFO: `/v3/users/2f7711aea6094976826571bf92746552` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:15.855550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7711aea6094976826571bf92746552)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:15.858063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:15.918024 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7711aea6094976826571bf92746552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:15.927689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7711aea6094976826571bf92746552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:15.940550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7711aea6094976826571bf92746552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:15.941329 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7711aea6094976826571bf92746552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:15.950420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7711aea6094976826571bf92746552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:15.957072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1141/2281] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:02:15 2022] DELETE /identity/v3/users/2f7711aea6094976826571bf92746552 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:02:15.961540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:15.962017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:15.962105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:15.970604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:15.983423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:15.984831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:16.051274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1141/2282] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:16.061136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:16.062574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:16.093982 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:16.126122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dbe3b-d7df-490a-a763-dededecbe7d0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:16.129764 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe42024-202b-45a4-80af-3126935666f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:16.130370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe42024-202b-45a4-80af-3126935666f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:16.130649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe42024-202b-45a4-80af-3126935666f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:16.130924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe42024-202b-45a4-80af-3126935666f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:16.140203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe42024-202b-45a4-80af-3126935666f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:16.141690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe42024-202b-45a4-80af-3126935666f5 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:16.151326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1142/2283] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:02:16.453829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:02:16.456114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:16.484613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ec8d6-86c6-4847-9589-f273a341b296 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:02:16.489547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:02:16.490188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:16.490469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:16.490766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] PATH_INFO: `/v3/projects/9a8274f4532c4076b47d3bc93e03270b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:16.491875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8274f4532c4076b47d3bc93e03270b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:02:16.493822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:02:16.542023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8274f4532c4076b47d3bc93e03270b', 'request_id': 'req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:16.553486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8274f4532c4076b47d3bc93e03270b', 'request_id': 'req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:16.563393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8274f4532c4076b47d3bc93e03270b', 'request_id': 'req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:16.579310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] The token cache is being invalidate because project 9a8274f4532c4076b47d3bc93e03270b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:02:16.579677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:16.580537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef3b315f-52ab-4ac5-bb9f-d7689d9f6ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:02:16.582099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1142/2284] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:02:16 2022] DELETE /identity/v3/projects/9a8274f4532c4076b47d3bc93e03270b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:02:26.147107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=14 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:26.574966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:02:40.658958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:02:40.659493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:02:40.659493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:02:40.678616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] MFA Rules not processed for user `cd22d2df9ced4bb0961596385854422d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:02:40.679817 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:02:40.735462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1143/2285] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:02:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:02:50.727945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:03:00.597973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:00.598502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:00.598693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:00.619244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:03:00.620483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:00.681214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1143/2286] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:03:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:03:00.723623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:00.725324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:00.821907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78810cf-6d60-4b28-8e67-995b8c724de8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:00.829516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999664c-be91-41d0-bf82-7839aa309860 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:00.830206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999664c-be91-41d0-bf82-7839aa309860 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:00.830522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999664c-be91-41d0-bf82-7839aa309860 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:00.830841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999664c-be91-41d0-bf82-7839aa309860 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:00.839478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999664c-be91-41d0-bf82-7839aa309860 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:00.841089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999664c-be91-41d0-bf82-7839aa309860 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:00.850785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1144/2287] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:03:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:03:05.895991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:05.898783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:05.985062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218fa7b-b406-4727-9f5b-abbef0a199a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:06.003008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:06.004579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:06.004950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:06.005720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:06.007653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:06.103996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:06.105784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a1f4c-c233-4cc4-822b-6092a384bfcf service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:06.117173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1144/2288] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:03:10.847229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=32 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:03:16.112248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=75 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:03:16.418575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d873d22-9e69-4884-a0d9-8a6fae7dbae3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.419117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d873d22-9e69-4884-a0d9-8a6fae7dbae3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.419117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d873d22-9e69-4884-a0d9-8a6fae7dbae3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.421096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1145/2289] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 13:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 13:03:16.427264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913653a-94f0-45f3-9e61-00feedb3d6c8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.427555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913653a-94f0-45f3-9e61-00feedb3d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.427834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913653a-94f0-45f3-9e61-00feedb3d6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.448786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f913653a-94f0-45f3-9e61-00feedb3d6c8 None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:03:16.450012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f913653a-94f0-45f3-9e61-00feedb3d6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:16.519716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1145/2290] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:03:16.532799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d873d22-9e69-4884-a0d9-8a6fae7dbae3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:16.534494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4d873d22-9e69-4884-a0d9-8a6fae7dbae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:16.592029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d873d22-9e69-4884-a0d9-8a6fae7dbae3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:16.595588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd2120-7163-49c0-81a9-5f3efd8dd061 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:16.596190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd2120-7163-49c0-81a9-5f3efd8dd061 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.596502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd2120-7163-49c0-81a9-5f3efd8dd061 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.596777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd2120-7163-49c0-81a9-5f3efd8dd061 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.606621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd2120-7163-49c0-81a9-5f3efd8dd061 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:16.608720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd2120-7163-49c0-81a9-5f3efd8dd061 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:16.619263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1146/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:03:16.690097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5606c9-eba2-4b30-8027-b63e97cfa70f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.690623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5606c9-eba2-4b30-8027-b63e97cfa70f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.690903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5606c9-eba2-4b30-8027-b63e97cfa70f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.692375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1146/2292] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 13:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 13:03:16.697187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf38c0-7736-4771-a823-f3d87eb1aa51 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.697554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf38c0-7736-4771-a823-f3d87eb1aa51 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.697842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf38c0-7736-4771-a823-f3d87eb1aa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.718466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e4cf38c0-7736-4771-a823-f3d87eb1aa51 None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:03:16.719674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e4cf38c0-7736-4771-a823-f3d87eb1aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:16.726125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1147/2293] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:03:16.738734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5606c9-eba2-4b30-8027-b63e97cfa70f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:16.740551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dd5606c9-eba2-4b30-8027-b63e97cfa70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:16.753998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5606c9-eba2-4b30-8027-b63e97cfa70f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:16.758577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5bba21-1953-425c-9720-82493e0fe372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:16.759266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bba21-1953-425c-9720-82493e0fe372 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.759547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bba21-1953-425c-9720-82493e0fe372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.759827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5bba21-1953-425c-9720-82493e0fe372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.767640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bba21-1953-425c-9720-82493e0fe372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:16.769372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5bba21-1953-425c-9720-82493e0fe372 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:16.780968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1147/2294] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:03:16.867314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0389f-a054-499d-9a8f-35e3161cbdff None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.867601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0389f-a054-499d-9a8f-35e3161cbdff None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.867876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0389f-a054-499d-9a8f-35e3161cbdff None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.869270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1148/2295] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 13:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 13:03:16.874877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4f7a0-8121-4f50-a52d-c0540c588995 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.875175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4f7a0-8121-4f50-a52d-c0540c588995 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.875473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4f7a0-8121-4f50-a52d-c0540c588995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.895443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-80b4f7a0-8121-4f50-a52d-c0540c588995 None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:03:16.896726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80b4f7a0-8121-4f50-a52d-c0540c588995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:16.906546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1148/2296] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:03:16.916794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0389f-a054-499d-9a8f-35e3161cbdff None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:16.928336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0389f-a054-499d-9a8f-35e3161cbdff None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:16.931868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:16.932492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:16.932773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:16.933042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:16.943153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:16.944620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:16.955461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1149/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:03:19.895699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:19.896177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:19.896315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:19.915157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:03:19.916513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:19.923834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1149/2298] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:03:19.937352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:19.948645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df321fbe-b038-458e-b96f-29b5be06f06f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:19.957928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:19.958579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:19.958846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:19.959115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:19.968117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:19.969724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:19.978508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1150/2299] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:03:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:03:27.150604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:27.160541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097234e-eff9-422f-98e8-6c686e065e44 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:27.164186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:27.164818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:27.165097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:27.165368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:27.167097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:27.181857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:27.183264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:27.195149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1150/2300] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:03:28.396847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:28.406250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377de95-5ddf-4667-ba4b-968d3085d68e service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:28.409901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:28.410496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:28.410765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:28.411038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:28.412794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:28.427625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:28.429092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:28.441787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1151/2301] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:03:37.189283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=49 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:03:38.435793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=47 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:03:47.326625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:47.328085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:47.341759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c144b-0951-49f4-8f77-bd23bab3095f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:47.347779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:47.348457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:47.348688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:47.348959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:47.350565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:47.403359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:47.404826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:47.412434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:47.414024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd615f-5e18-4e1a-8f85-65518d5a7355 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:47.420389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc254d-a558-45b9-92c1-d1c2faf3903e service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:47.427069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:47.427670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:47.427948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:47.428228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:47.429928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:47.432449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1151/2302] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:03:47.492659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:47.494282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:47.503941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1152/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:03:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:03:49.426607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:49.427040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:49.427163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:49.448150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:03:49.449387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:03:49.456554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1152/2304] 10.222.0.57 () {58 vars in 967 bytes} [Thu Sep 15 13:03:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:03:49.469381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:03:49.480547 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce48b0-3dfa-41db-8642-acc247506167 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:03:49.485890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe52c9-65c4-4e4a-9406-1f3058b94669 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:03:49.486476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe52c9-65c4-4e4a-9406-1f3058b94669 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:03:49.486750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe52c9-65c4-4e4a-9406-1f3058b94669 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:03:49.487018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe52c9-65c4-4e4a-9406-1f3058b94669 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:03:49.494703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe52c9-65c4-4e4a-9406-1f3058b94669 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:03:49.496264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe52c9-65c4-4e4a-9406-1f3058b94669 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:03:49.504748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1153/2305] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:03:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 13:03:59.441434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=31 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:03:59.501563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=34 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:04:08.288122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:08.299640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd705-a2b8-4348-8fd5-08c155be85b4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:08.303169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:08.303766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:08.304043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:08.304343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:08.306039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:08.316235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:08.317693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:08.325472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1153/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:10.435122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:10.435619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:10.435716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:10.455362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] MFA Rules not processed for user `e2ef1277bf944722aa33077985fb4734`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:04:10.456574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:10.463222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1154/2307] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:04:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:04:15.797524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:15.810682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b3b35-c64e-45fc-b0b0-81a8947b6f22 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:15.816786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:15.817415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:15.817707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:15.818021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:15.819998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:15.881215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:15.882809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:15.892051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1154/2308] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:16.561134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:16.574417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a545c05-c27d-447c-bba0-6b914ffbf0ff None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:16.582698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:16.583565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:16.584020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:16.584320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:16.586455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:16.599539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:16.601174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c49833-ae8a-4308-9344-eedb0b967a5b service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:16.612253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1155/2309] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:25.887668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=34 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:04:26.607076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=9 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:04:28.514902 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:28.526490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56201-c080-4975-aedd-1e57adf14af9 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:28.532624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:28.533226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:28.533507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:28.533777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:28.535488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:28.545640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:28.548080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:28.556969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1155/2310] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:38.552871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:04:41.408041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:41.408613 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:41.408996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:41.432840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] MFA Rules not processed for user `cd22d2df9ced4bb0961596385854422d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:04:41.434280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:41.441470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1156/2311] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:04:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:04:41.805725 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:41.815155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05c59d-7856-49e6-a04f-bb0fab3f552b service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:41.818723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:41.819337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:41.819622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:41.819895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:41.821663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:41.832082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:41.833572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:41.842032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1156/2312] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:42.164137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:42.165633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:42.238911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39b18c-538a-4c49-b00e-e76a24f48ac4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:42.243819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:42.244447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:42.244722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:42.244990 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:42.246642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:42.258762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:42.260357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:42.271150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1157/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:51.259283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:51.274298 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f975a-177d-42fe-91cf-d77281f8bbfc service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:51.279906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:51.280705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:51.280977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:51.281244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:51.283021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:51.295366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:51.297303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cafd8-0ebe-4ad4-a496-6fe38932014d service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:51.308181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1157/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:04:52.266693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=35 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:04:55.127766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:04:55.140713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340191-8356-4242-8286-57d6fba65250 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:04:55.146633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:04:55.147234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:04:55.147514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:04:55.147790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:04:55.149514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:04:55.161492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:04:55.163089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:04:55.174076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1158/2315] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:04:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:05:01.303709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=12 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:05.169134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:05.951449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:05.952769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:05.952769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:05.972149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] MFA Rules not processed for user `2f0feb7134ee408b989b398a315850fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:05.973374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:05.980785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1158/2316] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:05:06.284914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:06.294049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d4281-f9df-46be-9ef6-2dbe508d15d2 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:06.298438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:06.299019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:06.299295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:06.299565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:06.301281 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:06.310352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:06.312623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:06.320497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1159/2317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:15.965820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:16.316801 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:33.581015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:33.590549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86361d8f-61c5-422f-9d03-32b848299c56 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:33.594061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:33.594651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:33.594926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:33.595192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:33.597677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:33.660240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:33.661624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:33.671404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1159/2318] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:37.263396 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:37.264897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:37.280503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7d70a-5f0c-4cca-9219-9a3f37e15ba3 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:37.285891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:37.286498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.286779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.287056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.288783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:37.301987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:37.304584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:37.316944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:37.318016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1160/2319] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:37.320104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:37.345506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcaf728-652a-499d-b0d2-566a8386744a service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:37.351073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:37.351745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.352028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.352329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] PATH_INFO: `/v3/users/e2ef1277bf944722aa33077985fb4734` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.353456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ef1277bf944722aa33077985fb4734)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:37.355358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:37.403776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ef1277bf944722aa33077985fb4734'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.416931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ef1277bf944722aa33077985fb4734'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.424587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ef1277bf944722aa33077985fb4734'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.431920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ef1277bf944722aa33077985fb4734'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.432516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d261e4-a4d1-4a80-95ff-24eef3dc63cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ef1277bf944722aa33077985fb4734'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.438366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1160/2320] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:05:37 2022] DELETE /identity/v3/users/e2ef1277bf944722aa33077985fb4734 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:05:37.613025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:37.624830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a20a6-fc99-47d7-9b1c-603ca4b37d93 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:37.683570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:37.684120 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.684427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.684699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] PATH_INFO: `/v3/projects/ffff4b9feef3437cb5529aeaaacc3f86` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.685794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffff4b9feef3437cb5529aeaaacc3f86)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:37.687676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:37.732452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffff4b9feef3437cb5529aeaaacc3f86', 'request_id': 'req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.738100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffff4b9feef3437cb5529aeaaacc3f86', 'request_id': 'req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.743986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffff4b9feef3437cb5529aeaaacc3f86', 'request_id': 'req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.756642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] The token cache is being invalidate because project ffff4b9feef3437cb5529aeaaacc3f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:05:37.756934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.757561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ada7e3fc-9d5f-479e-b9fc-c1af487301c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:05:37.759042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1161/2321] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:05:37 2022] DELETE /identity/v3/projects/ffff4b9feef3437cb5529aeaaacc3f86 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:05:37.764127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.764442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.764720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.785212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:37.786425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:37.845468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1161/2322] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:05:37.853928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.854209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.854478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.874256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:37.875456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:37.883852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1162/2323] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:05:37.887971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:37.898592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e9d702-450e-4aed-b47d-79d0d607d8de None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:37.903208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:37.904068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.904461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.904745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.905383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:05:37.905689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:37.907393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:37.908997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:37.922084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1162/2324] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:05:37.927186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:37.936585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cab0e1-6b32-41e1-b155-651b0106c2ab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:37.941498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:37.942088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:37.942369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:37.942642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:37.943310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:37.945299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:37.977153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1163/2325] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:05:37 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:05:37.978537 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:37.989433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b9140-2950-4302-97b3-b646f7f6ebcb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.048078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.048908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.049189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.049462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.050089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.052004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.073132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1163/2326] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:05:37 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:05:38.077405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.088570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73509c9e-6ca6-46fb-9617-a5971fcdef89 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.094090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.094694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.094973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.095249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.096537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.098146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.108605 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1164/2327] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:05:38.112776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.122957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5dcf4-6aeb-4c14-b217-2bb0ff687f26 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.127912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531e3fa-fcff-4d63-9813-4cdc67008337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.128546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531e3fa-fcff-4d63-9813-4cdc67008337 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.128830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531e3fa-fcff-4d63-9813-4cdc67008337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.129108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531e3fa-fcff-4d63-9813-4cdc67008337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.129740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531e3fa-fcff-4d63-9813-4cdc67008337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.131330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531e3fa-fcff-4d63-9813-4cdc67008337 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.141748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1164/2328] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:05:38.146608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.157006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34a2a-7084-4f5b-b245-97a41924778d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.161783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.162423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.162704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.162979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] PATH_INFO: `/v3/projects/ebf212e88cc0401a99c43f9a1cf8b18f/users/4a44e69095bb49c2b9694225ca24b2d2/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.178558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebf212e88cc0401a99c43f9a1cf8b18f, user_id=4a44e69095bb49c2b9694225ca24b2d2, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.181359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.194271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1165/2329] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:05:38 2022] PUT /identity/v3/projects/ebf212e88cc0401a99c43f9a1cf8b18f/users/4a44e69095bb49c2b9694225ca24b2d2/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:05:38.205724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.206013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.206283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.235252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] MFA Rules not processed for user `4a44e69095bb49c2b9694225ca24b2d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:38.237049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:38.303999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1165/2330] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:05:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:05:38.308676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.324076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:38.374462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cd7ab-bb56-4ad0-a324-53b2472a15e5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.380192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.380889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.381262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.381541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.382175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.384083 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.417405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1166/2331] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:05:38 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:05:38.421364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.432968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c899ac-121c-4ee5-bc3a-19c940d6d70f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.489872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.490510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.490791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.491066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.491756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.493718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.503253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:38.515337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1166/2332] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:05:38 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:05:38.520515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.530837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107dc1f-a0bb-47a9-b7d2-90dc7fc982c4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.535488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.536077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.536387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.536661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.537300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.538913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.549930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1167/2333] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:05:38.554047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.563882 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fb6b2-fadc-47a9-ab09-f0e9e5815a61 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.569253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.569909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.570191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.570467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] PATH_INFO: `/v3/projects/e0a31a53380e4ed49f7986ef286428d7/users/b2f648f8f7244dc9960e84f7fb9dde01/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.586812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a31a53380e4ed49f7986ef286428d7, user_id=b2f648f8f7244dc9960e84f7fb9dde01, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.589525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.602144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1167/2334] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:05:38 2022] PUT /identity/v3/projects/e0a31a53380e4ed49f7986ef286428d7/users/b2f648f8f7244dc9960e84f7fb9dde01/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:05:38.606427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.617254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fe47-6c40-441d-a9a8-0f32a490d61f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.672139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb9daf-7e32-4b1f-9021-59bbb070d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.672778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb9daf-7e32-4b1f-9021-59bbb070d544 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.673068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb9daf-7e32-4b1f-9021-59bbb070d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.673340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb9daf-7e32-4b1f-9021-59bbb070d544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.674596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb9daf-7e32-4b1f-9021-59bbb070d544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.676193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb9daf-7e32-4b1f-9021-59bbb070d544 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.685751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1168/2335] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:05:38.690622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.700774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83260836-0729-43a6-8a75-0cf1eba3cdba admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.705478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.706124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.706406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.706682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] PATH_INFO: `/v3/projects/e0a31a53380e4ed49f7986ef286428d7/users/b2f648f8f7244dc9960e84f7fb9dde01/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.708592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a31a53380e4ed49f7986ef286428d7, user_id=b2f648f8f7244dc9960e84f7fb9dde01, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.711208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.721550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1168/2336] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:05:38 2022] PUT /identity/v3/projects/e0a31a53380e4ed49f7986ef286428d7/users/b2f648f8f7244dc9960e84f7fb9dde01/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:05:38.731607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.731897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.732176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.761146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] MFA Rules not processed for user `b2f648f8f7244dc9960e84f7fb9dde01`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:38.762296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:38.829809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1169/2337] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:05:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:05:38.841344 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:38.843082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:38.901143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c66f6-5c3c-4519-a9f1-31f7f1431894 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:38.904804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdab5f8-4216-4d33-993d-27228c92dbbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:38.905407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab5f8-4216-4d33-993d-27228c92dbbd service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:38.905690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab5f8-4216-4d33-993d-27228c92dbbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:38.905964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdab5f8-4216-4d33-993d-27228c92dbbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:38.915085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab5f8-4216-4d33-993d-27228c92dbbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:38.916644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdab5f8-4216-4d33-993d-27228c92dbbd service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:38.926601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1169/2338] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:39.002784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:39.012910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfc47-ca03-42b6-8f8f-694f0b9ba34e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:39.016637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa90048-3e2f-4a2c-bdd7-7e6b87bdf01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:39.017239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa90048-3e2f-4a2c-bdd7-7e6b87bdf01b service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:39.017798 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa90048-3e2f-4a2c-bdd7-7e6b87bdf01b service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:39.018081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa90048-3e2f-4a2c-bdd7-7e6b87bdf01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:39.023789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:39.075921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa90048-3e2f-4a2c-bdd7-7e6b87bdf01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:39.077401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa90048-3e2f-4a2c-bdd7-7e6b87bdf01b service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:39.086972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1170/2339] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:48.921794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:49.083139 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=24 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:05:51.738787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc84ca-4717-41cf-94e9-bf0f4eece7ef None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:51.739423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc84ca-4717-41cf-94e9-bf0f4eece7ef None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:51.739423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc84ca-4717-41cf-94e9-bf0f4eece7ef None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:51.740841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1170/2340] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 13:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 13:05:51.747755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff12a7-3316-4959-a3e0-4994a9060fc2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:51.748303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff12a7-3316-4959-a3e0-4994a9060fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:51.748645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff12a7-3316-4959-a3e0-4994a9060fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:51.772974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f7ff12a7-3316-4959-a3e0-4994a9060fc2 None None] MFA Rules not processed for user `53860f84e5cc4c84b15a8340f10c26d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:51.774253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f7ff12a7-3316-4959-a3e0-4994a9060fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:51.835373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1171/2341] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 13:05:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:05:51.859541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc84ca-4717-41cf-94e9-bf0f4eece7ef None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:51.861689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18bc84ca-4717-41cf-94e9-bf0f4eece7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:51.877385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc84ca-4717-41cf-94e9-bf0f4eece7ef None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:51.899058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08881a-d7c9-465d-84d0-4e42169f5db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:51.899785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08881a-d7c9-465d-84d0-4e42169f5db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:51.900099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08881a-d7c9-465d-84d0-4e42169f5db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:51.900438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08881a-d7c9-465d-84d0-4e42169f5db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:51.915173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08881a-d7c9-465d-84d0-4e42169f5db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:51.916852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08881a-d7c9-465d-84d0-4e42169f5db9 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:51.927208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1171/2342] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:52.020206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24691b1-85cd-41d7-87c1-e3123f14f476 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:52.020562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24691b1-85cd-41d7-87c1-e3123f14f476 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:52.020845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24691b1-85cd-41d7-87c1-e3123f14f476 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:52.022240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1172/2343] 10.222.0.57 () {54 vars in 921 bytes} [Thu Sep 15 13:05:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 13:05:52.028306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d186d6-30b7-4e61-9e4f-4de4a94be715 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:52.028610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d186d6-30b7-4e61-9e4f-4de4a94be715 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:52.028888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d186d6-30b7-4e61-9e4f-4de4a94be715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:52.048677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-68d186d6-30b7-4e61-9e4f-4de4a94be715 None None] MFA Rules not processed for user `03f7ea8f3fb64e3d96fbe277e07b5435`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:52.049798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-68d186d6-30b7-4e61-9e4f-4de4a94be715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:52.125217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1172/2344] 10.222.0.57 () {58 vars in 1018 bytes} [Thu Sep 15 13:05:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:05:52.136787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24691b1-85cd-41d7-87c1-e3123f14f476 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:05:52.148523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24691b1-85cd-41d7-87c1-e3123f14f476 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:05:52.152124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:05:52.152758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:52.153039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:52.153312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:52.163612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:05:52.165073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:05:52.176403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1173/2345] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:05:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:05:53.827329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1789da-abb6-4fc8-b754-bf7f21ea48dd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:05:53.827864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1789da-abb6-4fc8-b754-bf7f21ea48dd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:05:53.827978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1789da-abb6-4fc8-b754-bf7f21ea48dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:05:53.849086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7a1789da-abb6-4fc8-b754-bf7f21ea48dd None None] MFA Rules not processed for user `cff1145566f1408a9a22da9088581024`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:05:53.850307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7a1789da-abb6-4fc8-b754-bf7f21ea48dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:05:53.905335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1173/2346] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:05:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:02.171048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=29 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:03.898425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=62 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:25.004647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:25.064895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8076c0-aa86-486e-be2d-49a45063d1ad service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:25.069722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:25.070340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:25.070636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:25.070905 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] PATH_INFO: `/v3/users/4a44e69095bb49c2b9694225ca24b2d2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:25.072018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a44e69095bb49c2b9694225ca24b2d2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:25.073970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:25.119869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a44e69095bb49c2b9694225ca24b2d2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.126876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a44e69095bb49c2b9694225ca24b2d2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.138815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a44e69095bb49c2b9694225ca24b2d2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.139484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a44e69095bb49c2b9694225ca24b2d2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.148522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a44e69095bb49c2b9694225ca24b2d2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.154471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1174/2347] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:25 2022] DELETE /identity/v3/users/4a44e69095bb49c2b9694225ca24b2d2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:25.158962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:25.159412 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:25.159523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:25.179216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:25.180445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:25.238109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1174/2348] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:25.248580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:25.249992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:25.307939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9739ffa-285c-4a03-9986-347d0e65d37a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:25.311577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:25.312179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:25.312503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:25.312777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:25.322224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:25.323676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:25.333263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1175/2349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:25.602893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:25.604388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:25.617282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9b845-8676-44eb-a157-9e2cde54895d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:25.621779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:25.622419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:25.622701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:25.622995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] PATH_INFO: `/v3/projects/ebf212e88cc0401a99c43f9a1cf8b18f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:25.624148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebf212e88cc0401a99c43f9a1cf8b18f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:25.626092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:25.674084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebf212e88cc0401a99c43f9a1cf8b18f', 'request_id': 'req-de5f1188-3e53-4070-940c-722da37e07de'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.680263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebf212e88cc0401a99c43f9a1cf8b18f', 'request_id': 'req-de5f1188-3e53-4070-940c-722da37e07de'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.686603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebf212e88cc0401a99c43f9a1cf8b18f', 'request_id': 'req-de5f1188-3e53-4070-940c-722da37e07de'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.698820 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] The token cache is being invalidate because project ebf212e88cc0401a99c43f9a1cf8b18f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:25.699131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.699756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.701260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1175/2350] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:25 2022] DELETE /identity/v3/projects/ebf212e88cc0401a99c43f9a1cf8b18f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:25.705403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:25.715326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e4d8f-68ff-4201-9c90-8b308322fd56 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:25.733844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:25.773937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:25.774579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:25.774863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:25.775138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] PATH_INFO: `/v3/users/b2f648f8f7244dc9960e84f7fb9dde01` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:25.776288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f648f8f7244dc9960e84f7fb9dde01)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:25.778354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:25.823581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f648f8f7244dc9960e84f7fb9dde01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.831227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f648f8f7244dc9960e84f7fb9dde01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.843058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f648f8f7244dc9960e84f7fb9dde01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.843737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f648f8f7244dc9960e84f7fb9dde01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.850331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c770a65-dd83-4c00-a999-fc185e6c2627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f648f8f7244dc9960e84f7fb9dde01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:25.856131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1176/2351] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:25 2022] DELETE /identity/v3/users/b2f648f8f7244dc9960e84f7fb9dde01 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:26.131870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.133366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:26.148067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f1188-3e53-4070-940c-722da37e07de admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.207577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.208437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.208708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.208981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] PATH_INFO: `/v3/projects/e0a31a53380e4ed49f7986ef286428d7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.210095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a31a53380e4ed49f7986ef286428d7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.211995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.241251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:26.260517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a31a53380e4ed49f7986ef286428d7', 'request_id': 'req-1eefe447-8a79-453d-abe7-f68bc106bad0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:26.266172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a31a53380e4ed49f7986ef286428d7', 'request_id': 'req-1eefe447-8a79-453d-abe7-f68bc106bad0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:26.273654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a31a53380e4ed49f7986ef286428d7', 'request_id': 'req-1eefe447-8a79-453d-abe7-f68bc106bad0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:26.284763 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] The token cache is being invalidate because project e0a31a53380e4ed49f7986ef286428d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:26.285102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:26.286016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1eefe447-8a79-453d-abe7-f68bc106bad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:26.287303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1176/2352] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:26 2022] DELETE /identity/v3/projects/e0a31a53380e4ed49f7986ef286428d7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:26.292144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.292604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.292796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.315451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:26.317511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:26.374642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1177/2353] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:26.383277 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3b30b-267a-4521-807a-03906a86812e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.383568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3b30b-267a-4521-807a-03906a86812e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.383843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3b30b-267a-4521-807a-03906a86812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.405655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12c3b30b-267a-4521-807a-03906a86812e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:26.406892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12c3b30b-267a-4521-807a-03906a86812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:26.415261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1177/2354] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:26.419503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.429277 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b52c61-de8c-445a-a9fe-e7180434a111 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.433819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.434415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.434698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.434972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.435638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:06:26.435970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.437623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.449720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1178/2355] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:06:26.453849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3b30b-267a-4521-807a-03906a86812e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.464199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3b30b-267a-4521-807a-03906a86812e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.468724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.469312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.469588 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.469855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.470483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.472397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.499773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1178/2356] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:06:26.504524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.514333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25580-3987-455d-84c8-48ef304fe805 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.540187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:26.574102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.574703 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.574979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.575262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.575883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.577833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.598472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1179/2357] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/users => generated 279 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:26.602631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.613048 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2c319-cd27-4c5f-8d55-9f1daeb888ab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.618221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.618808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.619084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.619353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.619976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.621592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.632524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1179/2358] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:26.636361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.646359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c3efb8-55c1-481d-b8cb-bce26019f14f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.650925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f625-28de-48d4-935e-b0ba514b3795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.651512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f625-28de-48d4-935e-b0ba514b3795 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.651790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f625-28de-48d4-935e-b0ba514b3795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.652063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f625-28de-48d4-935e-b0ba514b3795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.652743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1f625-28de-48d4-935e-b0ba514b3795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.654325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1f625-28de-48d4-935e-b0ba514b3795 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.664935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1180/2359] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:26.668922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.679523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fa223-87e5-41d3-a33c-b9cca57b3158 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.684059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.684848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.685164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.685462 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] PATH_INFO: `/v3/projects/47d53ec5173546c2b7cb83514ad36898/users/cfc59443405747c594d78065cb470803/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.701000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d53ec5173546c2b7cb83514ad36898, user_id=cfc59443405747c594d78065cb470803, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.703620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.716711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1180/2360] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:26 2022] PUT /identity/v3/projects/47d53ec5173546c2b7cb83514ad36898/users/cfc59443405747c594d78065cb470803/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:26.727192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.727531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.727809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.755927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] MFA Rules not processed for user `cfc59443405747c594d78065cb470803`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:26.757174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:26.821100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1181/2361] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:26.825643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.887526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04fa83-8b8b-466d-9a32-b0627ac384eb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.892195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.892830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.893115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.893389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.894023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.896185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:26.922753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1181/2362] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/projects => generated 357 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:06:26.927529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:26.938103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4cd30-0fd6-4ac0-a83e-e1066b3d7005 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:26.994119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:26.994644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:26.994957 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:26.995242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:26.995863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:26.997872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.018818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1182/2363] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:26 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:27.023289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:27.033508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0f76-f397-4be4-94de-c0c2ac6dea40 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:27.038731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:27.039329 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.039614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.039886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.040554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:27.042144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.053195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1182/2364] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:27.057321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:27.067267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dc377-a6e7-4342-b96e-6eb794ba218f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:27.071900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:27.072578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.072861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.073133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] PATH_INFO: `/v3/projects/d35db74e14284f93ada12e1295c93df8/users/224f518e95ab4b3693a8e57248d3f6af/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.087963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35db74e14284f93ada12e1295c93df8, user_id=224f518e95ab4b3693a8e57248d3f6af, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:27.090669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.104931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1183/2365] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:27 2022] PUT /identity/v3/projects/d35db74e14284f93ada12e1295c93df8/users/224f518e95ab4b3693a8e57248d3f6af/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:27.109028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:27.119971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3ab1d-06fc-4143-bfb6-a0bfab014a00 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:27.139667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:27.178018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dbafd-df45-404f-a99b-02640061e880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:27.178629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dbafd-df45-404f-a99b-02640061e880 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.178913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dbafd-df45-404f-a99b-02640061e880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.179186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dbafd-df45-404f-a99b-02640061e880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.179818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1dbafd-df45-404f-a99b-02640061e880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:27.181443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1dbafd-df45-404f-a99b-02640061e880 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.191017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1183/2366] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:27.195231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:27.200629 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:27.206647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907def4-e47e-4953-a1ca-38cf3d3598b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:27.211616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:27.212286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.212709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.212986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] PATH_INFO: `/v3/projects/d35db74e14284f93ada12e1295c93df8/users/224f518e95ab4b3693a8e57248d3f6af/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.214875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d35db74e14284f93ada12e1295c93df8, user_id=224f518e95ab4b3693a8e57248d3f6af, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:27.217509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.228249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1184/2367] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:27 2022] PUT /identity/v3/projects/d35db74e14284f93ada12e1295c93df8/users/224f518e95ab4b3693a8e57248d3f6af/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:27.239812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.240320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.240447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.267536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] MFA Rules not processed for user `224f518e95ab4b3693a8e57248d3f6af`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:27.268724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:27.336777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1184/2368] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:27.349294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:27.352705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:27.411169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafcb83-b7a7-41c0-be7e-a0690aa4a13d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:27.414825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:27.415432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.415734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.416005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.426792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:27.428600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.438649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1185/2369] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:27.515712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:27.525784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2264d-8a4a-40e0-b750-39afdf4ca5e2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:27.529336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112b7d-2c32-476b-b1da-0426b8ad1411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:27.529933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112b7d-2c32-476b-b1da-0426b8ad1411 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:27.530215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112b7d-2c32-476b-b1da-0426b8ad1411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:27.530487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112b7d-2c32-476b-b1da-0426b8ad1411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:27.586101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db112b7d-2c32-476b-b1da-0426b8ad1411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:27.587527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db112b7d-2c32-476b-b1da-0426b8ad1411 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:27.596307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1185/2370] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:37.434471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:37.592265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=80 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:43.490688 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:43.595788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285f4aa-becc-4ede-a858-8b3dd52d0131 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:43.601323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:43.601875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:43.602718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:43.603006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] PATH_INFO: `/v3/users/cfc59443405747c594d78065cb470803` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:43.604177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc59443405747c594d78065cb470803)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:43.606142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:43.650377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc59443405747c594d78065cb470803'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:43.657087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc59443405747c594d78065cb470803'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:43.669205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc59443405747c594d78065cb470803'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:43.669866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc59443405747c594d78065cb470803'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:43.676534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc59443405747c594d78065cb470803'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:43.682427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1186/2371] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:43 2022] DELETE /identity/v3/users/cfc59443405747c594d78065cb470803 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:43.687065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:43.687546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:43.687813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:43.736281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:43.737496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:43.796198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1186/2372] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:43.806052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:43.807446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:43.849229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:43.881878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dec8e4-ed74-433f-87b9-076ebbd6ac46 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:43.886317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:43.886926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:43.887204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:43.887475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:43.896654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:43.899011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:43.909933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1187/2373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:43.916975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:43.918555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:44.030165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c832c-bbaf-41da-8ae2-dd7fa756781d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:44.035293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:44.035911 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:44.036192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:44.036496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:44.038440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:44.052033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:44.054669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:44.066147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1187/2374] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:44.198799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:44.201076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:44.215345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846add7-113c-4bdb-9293-33f228fd11f7 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:44.220740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:44.221561 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:44.221842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:44.222125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] PATH_INFO: `/v3/projects/47d53ec5173546c2b7cb83514ad36898` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:44.223256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d53ec5173546c2b7cb83514ad36898)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:44.227025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:44.286339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d53ec5173546c2b7cb83514ad36898', 'request_id': 'req-96aae142-2b09-443d-916e-63ca62f597a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.294077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d53ec5173546c2b7cb83514ad36898', 'request_id': 'req-96aae142-2b09-443d-916e-63ca62f597a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.302928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d53ec5173546c2b7cb83514ad36898', 'request_id': 'req-96aae142-2b09-443d-916e-63ca62f597a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.317208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] The token cache is being invalidate because project 47d53ec5173546c2b7cb83514ad36898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:44.317511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.318145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.320483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1188/2375] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:44 2022] DELETE /identity/v3/projects/47d53ec5173546c2b7cb83514ad36898 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:44.324667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:44.338226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ce05-973d-4467-aa52-dfc7b00495ac service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:44.387373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:44.405039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:44.405687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:44.405993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:44.406270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] PATH_INFO: `/v3/users/224f518e95ab4b3693a8e57248d3f6af` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:44.407429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] RBAC: Authorizing `identity:delete_user(user_id=224f518e95ab4b3693a8e57248d3f6af)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:44.409596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:44.461530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224f518e95ab4b3693a8e57248d3f6af'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.481879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224f518e95ab4b3693a8e57248d3f6af'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.492425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224f518e95ab4b3693a8e57248d3f6af'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.505386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224f518e95ab4b3693a8e57248d3f6af'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.506122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-07b73529-8173-4d24-9e1b-9f2f78e5041d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224f518e95ab4b3693a8e57248d3f6af'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.515357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1188/2376] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:44 2022] DELETE /identity/v3/users/224f518e95ab4b3693a8e57248d3f6af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:06:44.813642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:44.815141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:44.829156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aae142-2b09-443d-916e-63ca62f597a6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:44.893354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:44.893899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:44.894181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:44.894456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] PATH_INFO: `/v3/projects/d35db74e14284f93ada12e1295c93df8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:44.895942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d35db74e14284f93ada12e1295c93df8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:44.898778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:44.955958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd35db74e14284f93ada12e1295c93df8', 'request_id': 'req-7d419695-09bc-44b3-ae26-160260d8c41d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.957218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:44.963961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd35db74e14284f93ada12e1295c93df8', 'request_id': 'req-7d419695-09bc-44b3-ae26-160260d8c41d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.971384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd35db74e14284f93ada12e1295c93df8', 'request_id': 'req-7d419695-09bc-44b3-ae26-160260d8c41d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.984648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] The token cache is being invalidate because project d35db74e14284f93ada12e1295c93df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:44.985021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.985652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7d419695-09bc-44b3-ae26-160260d8c41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:44.987456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1189/2377] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:44 2022] DELETE /identity/v3/projects/d35db74e14284f93ada12e1295c93df8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:44.993162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:44.993981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:44.993981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.050260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:45.051481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:45.115109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1189/2378] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:45.122889 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.123181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.123461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.144692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:45.146184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:45.155743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1190/2379] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:45.160425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.172155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472fd4c-0e62-45ed-9b5c-53a824841684 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.178193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.178872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.179163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.179508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.180081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:06:45.180452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.182012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.197335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1190/2380] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:06:45.201941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.213924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3458e9-ee51-4660-af72-348481c2018c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.218966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.219788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.220236 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.220573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.221179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.223075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.252748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1191/2381] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:45 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:06:45.257622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.274850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286c5c3-2b72-42c3-9aad-f19998368c73 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.302842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:45.344038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.344038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.344770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.344770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.345370 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.348126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.384316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1191/2382] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:45 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:45.388521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.405352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c168cd2-b90d-4111-8c98-1be825990303 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.410223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.411019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.411303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.411580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.412841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.414647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.427285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1192/2383] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:45.431761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.446913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b660fef-774a-4d73-8027-c4f37a123ad8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.452231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eaafaa-fc62-4c22-a55f-2f127ff3be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.452891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaafaa-fc62-4c22-a55f-2f127ff3be5d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.453182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaafaa-fc62-4c22-a55f-2f127ff3be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.453457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eaafaa-fc62-4c22-a55f-2f127ff3be5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.454104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eaafaa-fc62-4c22-a55f-2f127ff3be5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.455740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eaafaa-fc62-4c22-a55f-2f127ff3be5d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.469735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1192/2384] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:06:45.473862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.485792 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9bd30-11c0-4180-a299-d3c51cdcc8dd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.493553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.494614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.494760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.495054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] PATH_INFO: `/v3/projects/73994c369cd2421687cd6086333e9900/users/42e22a4d3e8e4864a123e16321a0688b/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.513742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] RBAC: Authorizing `identity:create_grant(project_id=73994c369cd2421687cd6086333e9900, user_id=42e22a4d3e8e4864a123e16321a0688b, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.516525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.534331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1193/2385] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:45 2022] PUT /identity/v3/projects/73994c369cd2421687cd6086333e9900/users/42e22a4d3e8e4864a123e16321a0688b/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:06:45.546374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.546787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.547136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.584643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] MFA Rules not processed for user `42e22a4d3e8e4864a123e16321a0688b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:45.586136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:45.665866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1193/2386] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:45.672202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.748190 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e206-9557-4ec6-801c-7a675e53602a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.754400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.755217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.755499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.755775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.756468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.759081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.792635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1194/2387] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:45 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:45.797830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.812483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223ef82-32ec-4b04-8b56-59a507bd23ae None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.879014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.879705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.880171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.880869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.881876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.884172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.920576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1194/2388] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:45 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:45.924283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.936555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a008a9-26a8-4eb3-8e0e-69102bd8c147 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.941559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.942262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.943014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.943014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:45.943604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:45.946129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:45.961937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1195/2389] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:06:45.962879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:45.977870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69624-bf13-4010-b61b-3390609cc719 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:45.983274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:45.983999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:45.984301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:45.984602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] PATH_INFO: `/v3/projects/72539ab917254493b63218056eb32d20/users/bc919cde2aac418ba7d6ff2a7aac568f/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:46.003148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72539ab917254493b63218056eb32d20, user_id=bc919cde2aac418ba7d6ff2a7aac568f, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:46.006015 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:46.028207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:46.033847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1195/2390] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:45 2022] PUT /identity/v3/projects/72539ab917254493b63218056eb32d20/users/bc919cde2aac418ba7d6ff2a7aac568f/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:06:46.037765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:46.064175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f0f67-547d-434c-891a-e45cd4a98e2b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:46.121428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:46.136720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a77342-a7e4-4025-b58e-9cbd484329b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:46.137331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77342-a7e4-4025-b58e-9cbd484329b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:46.137610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77342-a7e4-4025-b58e-9cbd484329b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:46.137877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77342-a7e4-4025-b58e-9cbd484329b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:46.138511 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a77342-a7e4-4025-b58e-9cbd484329b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:46.140125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a77342-a7e4-4025-b58e-9cbd484329b1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:46.151813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1196/2391] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:46.155233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:46.165821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4465804-e23a-4eb2-96ed-a76fb63911c0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:46.171009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:46.171754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:46.172109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:46.172423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] PATH_INFO: `/v3/projects/72539ab917254493b63218056eb32d20/users/bc919cde2aac418ba7d6ff2a7aac568f/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:46.174492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72539ab917254493b63218056eb32d20, user_id=bc919cde2aac418ba7d6ff2a7aac568f, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:46.177333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:46.188194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1196/2392] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:46 2022] PUT /identity/v3/projects/72539ab917254493b63218056eb32d20/users/bc919cde2aac418ba7d6ff2a7aac568f/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:06:46.199267 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:46.199594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:46.200027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:46.228915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] MFA Rules not processed for user `bc919cde2aac418ba7d6ff2a7aac568f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:46.230041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:46.296092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1197/2393] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:46.307863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:46.309654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:46.366025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b37d9-d438-4b9d-acd3-de337d13ae50 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:46.370429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e92c2-4750-4a79-92cd-730a5babc914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:46.371034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e92c2-4750-4a79-92cd-730a5babc914 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:46.371313 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e92c2-4750-4a79-92cd-730a5babc914 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:46.371584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e92c2-4750-4a79-92cd-730a5babc914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:46.380625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e92c2-4750-4a79-92cd-730a5babc914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:46.382802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e92c2-4750-4a79-92cd-730a5babc914 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:46.392183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1197/2394] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:47.767144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:47.829021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134a426-2110-4bd0-b091-5314e0ff7dde None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:47.834321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:47.835498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:47.835775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:47.836046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] PATH_INFO: `/v3/users/42e22a4d3e8e4864a123e16321a0688b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:47.837196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e22a4d3e8e4864a123e16321a0688b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:47.840095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:47.887354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e22a4d3e8e4864a123e16321a0688b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:47.894499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e22a4d3e8e4864a123e16321a0688b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:47.906147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:47.907542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e22a4d3e8e4864a123e16321a0688b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:47.908312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e22a4d3e8e4864a123e16321a0688b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:47.916233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e22a4d3e8e4864a123e16321a0688b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:47.922131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1198/2395] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:47 2022] DELETE /identity/v3/users/42e22a4d3e8e4864a123e16321a0688b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:47.926526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:47.927025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:47.927143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:47.947680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:47.949078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:48.012532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1198/2396] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:48.023498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:48.025008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:48.084580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03986a-8e54-4a69-a126-600f9f87d814 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:48.088939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:48.089576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:48.089857 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:48.090131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:48.100359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:48.101909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:48.112100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1199/2397] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:06:48.536217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:48.537710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:48.571779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20218b-4970-427c-a08b-4cdd3aba1e1d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:48.587638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:48.588321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:48.588607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:48.588878 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] PATH_INFO: `/v3/projects/73994c369cd2421687cd6086333e9900` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:48.590042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] RBAC: Authorizing `identity:delete_project(project_id=73994c369cd2421687cd6086333e9900)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:48.592025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:48.640402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73994c369cd2421687cd6086333e9900', 'request_id': 'req-544fe0df-0f57-4dc6-ac54-8fb1268c205d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.646112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73994c369cd2421687cd6086333e9900', 'request_id': 'req-544fe0df-0f57-4dc6-ac54-8fb1268c205d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.653032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73994c369cd2421687cd6086333e9900', 'request_id': 'req-544fe0df-0f57-4dc6-ac54-8fb1268c205d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.666998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] The token cache is being invalidate because project 73994c369cd2421687cd6086333e9900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:48.667357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.668027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.669554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1199/2398] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:48 2022] DELETE /identity/v3/projects/73994c369cd2421687cd6086333e9900 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:48.673620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:48.683959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93775a5-ae8c-4676-96fd-76ff0ef98a72 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:48.741302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:48.741934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:48.742207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:48.742478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] PATH_INFO: `/v3/users/bc919cde2aac418ba7d6ff2a7aac568f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:48.743585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc919cde2aac418ba7d6ff2a7aac568f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:48.745539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:48.792729 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc919cde2aac418ba7d6ff2a7aac568f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.799966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc919cde2aac418ba7d6ff2a7aac568f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.813180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc919cde2aac418ba7d6ff2a7aac568f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.813932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc919cde2aac418ba7d6ff2a7aac568f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.821985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-57106b74-34ab-4fa9-aca2-8d3616414cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc919cde2aac418ba7d6ff2a7aac568f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:48.828722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1200/2399] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:48 2022] DELETE /identity/v3/users/bc919cde2aac418ba7d6ff2a7aac568f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:49.256243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.258409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:49.282047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fe0df-0f57-4dc6-ac54-8fb1268c205d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.296918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:49.389190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.390572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.390939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.391364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] PATH_INFO: `/v3/projects/72539ab917254493b63218056eb32d20` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.392623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=72539ab917254493b63218056eb32d20)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.394628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:49.468796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72539ab917254493b63218056eb32d20', 'request_id': 'req-74635f0a-0d74-459e-aa18-5898fff5a74a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:49.476131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72539ab917254493b63218056eb32d20', 'request_id': 'req-74635f0a-0d74-459e-aa18-5898fff5a74a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:49.485017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72539ab917254493b63218056eb32d20', 'request_id': 'req-74635f0a-0d74-459e-aa18-5898fff5a74a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:49.500091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] The token cache is being invalidate because project 72539ab917254493b63218056eb32d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:49.500535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:49.501263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-74635f0a-0d74-459e-aa18-5898fff5a74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:49.503404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1200/2400] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:49 2022] DELETE /identity/v3/projects/72539ab917254493b63218056eb32d20 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:06:49.509156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.509448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.509748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.531030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:49.532458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:49.538720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:49.617705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1201/2401] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:49.627409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.627713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.627969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.665842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:49.669407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:49.677732 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1201/2402] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:49.682275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.692348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dac60a-fee0-486b-809f-3aa4b938a5fc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.698633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.699774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.700054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.700404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.701070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:06:49.701367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.702973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:49.718630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1202/2403] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:06:49.723109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.733881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ce1da-3666-45ba-9bd2-1c7e119d599d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.738646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.739247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.739530 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.739801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.740488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.742650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:49.773339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1202/2404] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:49 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:06:49.777602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.787650 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d54d66-c7c3-4600-a2b4-e4ded2805b63 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.847124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.847802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.847999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.848292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.848968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.851570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:49.879556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1203/2405] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:49 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:49.881099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.891790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56208f-2fb4-4319-817f-df08a670bb40 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.897321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.898144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.898425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.898698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.899340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.901139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:49.911564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1203/2406] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:49.916217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.926065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa469a-86a7-4ac7-8bb4-dfd0baca8bd1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.931345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce306-f582-48b5-8268-38a6d5e0265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.931944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce306-f582-48b5-8268-38a6d5e0265e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.932303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce306-f582-48b5-8268-38a6d5e0265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.932607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce306-f582-48b5-8268-38a6d5e0265e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.933242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bce306-f582-48b5-8268-38a6d5e0265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.934836 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bce306-f582-48b5-8268-38a6d5e0265e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:49.953038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1204/2407] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:49.954143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:49.965451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f00628-f8bf-490a-8911-de4e55e4165b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:49.970603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:49.971463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:49.971808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:49.972146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] PATH_INFO: `/v3/projects/6a38f5e73fea4f1d95a0033a14cc785d/users/135647a760e342b1a8c98cf7fac4dd8b/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:49.989832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a38f5e73fea4f1d95a0033a14cc785d, user_id=135647a760e342b1a8c98cf7fac4dd8b, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:49.993294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.010188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1204/2408] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:49 2022] PUT /identity/v3/projects/6a38f5e73fea4f1d95a0033a14cc785d/users/135647a760e342b1a8c98cf7fac4dd8b/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:50.020912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.021205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.021514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.050959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] MFA Rules not processed for user `135647a760e342b1a8c98cf7fac4dd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:50.052383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:50.141509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1205/2409] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:50.146843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.192264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:50.217592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b033f-ba44-4847-ac76-dba84e0e3706 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.222321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.222939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.223222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.223498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.224150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.226481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.254755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1205/2410] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:50 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:50.259903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.271411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b504d-a233-4356-8a6c-ca40d059a6c9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.275307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:50.332853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.333462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.333738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.334007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.334643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.337143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.359860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1206/2411] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:50 2022] POST /identity/v3/users => generated 306 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:06:50.363867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.376147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46c7c-78b1-4cf1-ab3d-b95ed623c577 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.380739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.381339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.381631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.382478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.382609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.384325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.400833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1206/2412] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:50.407684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.420071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6587194-05bd-4e9a-92d0-3a2effb52f00 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.425340 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.426020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.426342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.426754 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] PATH_INFO: `/v3/projects/91e5b762285448c99f629b8f2f31cfb6/users/07c0a69937304a0d89279a1aedb27d48/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.442400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e5b762285448c99f629b8f2f31cfb6, user_id=07c0a69937304a0d89279a1aedb27d48, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.446015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.460732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1207/2413] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:50 2022] PUT /identity/v3/projects/91e5b762285448c99f629b8f2f31cfb6/users/07c0a69937304a0d89279a1aedb27d48/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:50.465229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.475006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68101e7-edd2-48ab-a719-8a3a048c14b3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.531813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2724f2e-29c4-4619-8db6-ce14ce662f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.532448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2724f2e-29c4-4619-8db6-ce14ce662f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.532731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2724f2e-29c4-4619-8db6-ce14ce662f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.533006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2724f2e-29c4-4619-8db6-ce14ce662f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.533629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2724f2e-29c4-4619-8db6-ce14ce662f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.535247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2724f2e-29c4-4619-8db6-ce14ce662f52 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.545000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1207/2414] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:50.549087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.559244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7900fa2-1ee6-44e8-990a-b3a141d62e24 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.563772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.564447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.564726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.564996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] PATH_INFO: `/v3/projects/91e5b762285448c99f629b8f2f31cfb6/users/07c0a69937304a0d89279a1aedb27d48/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.566849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e5b762285448c99f629b8f2f31cfb6, user_id=07c0a69937304a0d89279a1aedb27d48, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.570056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.579949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1208/2415] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:50 2022] PUT /identity/v3/projects/91e5b762285448c99f629b8f2f31cfb6/users/07c0a69937304a0d89279a1aedb27d48/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:50.590394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.590686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.590995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.620112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] MFA Rules not processed for user `07c0a69937304a0d89279a1aedb27d48`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:50.621310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:50.693110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1208/2416] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:50.704167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.706038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:50.764522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b51f3d-1790-4006-b8ca-06b028fd3e18 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.768253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346141d5-4ce2-49ac-9745-a839aa00e7df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.768976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346141d5-4ce2-49ac-9745-a839aa00e7df service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.769273 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346141d5-4ce2-49ac-9745-a839aa00e7df service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.769544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346141d5-4ce2-49ac-9745-a839aa00e7df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.779656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346141d5-4ce2-49ac-9745-a839aa00e7df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.781125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346141d5-4ce2-49ac-9745-a839aa00e7df service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.793033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1209/2417] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:50.871429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:50.881293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dec9d-620e-40a9-832b-3fcbba70d452 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:50.885375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:50.886092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:50.886377 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:50.886650 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:50.923038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:50.960074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:50.961690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:50.973368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1209/2418] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:52.658459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec78e3d-30aa-4c5c-be3c-6bda93c71724 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:52.658459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec78e3d-30aa-4c5c-be3c-6bda93c71724 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:52.659101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec78e3d-30aa-4c5c-be3c-6bda93c71724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:52.679765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0ec78e3d-30aa-4c5c-be3c-6bda93c71724 None None] MFA Rules not processed for user `cd22d2df9ced4bb0961596385854422d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:52.681156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0ec78e3d-30aa-4c5c-be3c-6bda93c71724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:52.750987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1210/2419] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:55.107491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:55.172226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825d055-95d3-49db-83a0-6087ed4eac14 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:55.176788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:55.177416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:55.177691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:55.177960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] PATH_INFO: `/v3/users/135647a760e342b1a8c98cf7fac4dd8b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:55.179057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] RBAC: Authorizing `identity:delete_user(user_id=135647a760e342b1a8c98cf7fac4dd8b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:55.181189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:55.229804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135647a760e342b1a8c98cf7fac4dd8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:55.242230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135647a760e342b1a8c98cf7fac4dd8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:55.250699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135647a760e342b1a8c98cf7fac4dd8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:55.260737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135647a760e342b1a8c98cf7fac4dd8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:55.261391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135647a760e342b1a8c98cf7fac4dd8b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:55.267811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1210/2420] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:55 2022] DELETE /identity/v3/users/135647a760e342b1a8c98cf7fac4dd8b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:55.272516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:55.272983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:55.273344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:55.287121 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:55.294008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:55.295487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:55.364557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1211/2421] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:06:55.377594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:55.379075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:55.446455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:55.481693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60727b-6429-435d-b1ee-ea3abc49effa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:55.486852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:55.487701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:55.488052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:55.488464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:55.511779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:55.513513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:55.531074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1211/2422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:56.307679 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:56.309156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:56.322535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ae775-8080-4a96-896f-2094ab80a159 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:56.327893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:56.328569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:56.328796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:56.329196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] PATH_INFO: `/v3/projects/6a38f5e73fea4f1d95a0033a14cc785d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:56.330450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a38f5e73fea4f1d95a0033a14cc785d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:56.332473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:56.387894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a38f5e73fea4f1d95a0033a14cc785d', 'request_id': 'req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.393814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a38f5e73fea4f1d95a0033a14cc785d', 'request_id': 'req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.401172 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a38f5e73fea4f1d95a0033a14cc785d', 'request_id': 'req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.417456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] The token cache is being invalidate because project 6a38f5e73fea4f1d95a0033a14cc785d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:56.418169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.419913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.422452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1212/2423] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:56 2022] DELETE /identity/v3/projects/6a38f5e73fea4f1d95a0033a14cc785d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:56.426752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:56.441653 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebea57-0804-4816-b2e2-671cb5e622f8 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:56.513635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:56.514703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:56.515429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:56.515934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] PATH_INFO: `/v3/users/07c0a69937304a0d89279a1aedb27d48` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:56.517393 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c0a69937304a0d89279a1aedb27d48)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:56.519620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:56.575387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c0a69937304a0d89279a1aedb27d48'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.589390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c0a69937304a0d89279a1aedb27d48'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.596995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c0a69937304a0d89279a1aedb27d48'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.605876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c0a69937304a0d89279a1aedb27d48'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.606453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3ef86eaa-f508-45f4-9d70-9c2d2522485b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c0a69937304a0d89279a1aedb27d48'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:56.617877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1212/2424] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:06:56 2022] DELETE /identity/v3/users/07c0a69937304a0d89279a1aedb27d48 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:06:57.114366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.115832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:57.129199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b26aa-7f22-4fb8-80d3-bdc64a42bc4a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.214745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.215697 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.216030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.216453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] PATH_INFO: `/v3/projects/91e5b762285448c99f629b8f2f31cfb6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.217855 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e5b762285448c99f629b8f2f31cfb6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.220067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.260485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:57.277683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e5b762285448c99f629b8f2f31cfb6', 'request_id': 'req-7bc2c115-a0eb-4a36-a762-1309ee77ba42'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:57.285354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e5b762285448c99f629b8f2f31cfb6', 'request_id': 'req-7bc2c115-a0eb-4a36-a762-1309ee77ba42'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:57.297107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e5b762285448c99f629b8f2f31cfb6', 'request_id': 'req-7bc2c115-a0eb-4a36-a762-1309ee77ba42'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:57.310380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] The token cache is being invalidate because project 91e5b762285448c99f629b8f2f31cfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:06:57.311023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:57.311436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7bc2c115-a0eb-4a36-a762-1309ee77ba42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:06:57.313166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1213/2425] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:06:57 2022] DELETE /identity/v3/projects/91e5b762285448c99f629b8f2f31cfb6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:57.318003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.318292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.318572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.340754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:57.341970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:57.393353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:57.406983 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1213/2426] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:06:57.471756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.472175 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.472487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.491780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:57.493041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:57.500705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1214/2427] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:57.504924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.515289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff337d-8696-419a-9547-9c77ddbc31ee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.519847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.520733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.521028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.521301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.522008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:06:57.522315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.524589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.536309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1214/2428] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:06:57.540963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.549873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b089f9-1fb1-4a51-ad96-4ff5cc7ac52b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.554973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.555614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.555893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.556163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.556834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.558711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.586437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1215/2429] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:57 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:57.590494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.600516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ef1ef-a2aa-4325-9286-f4c35c009ed6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.659873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.660431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.660717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.660991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.661614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.663528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.685734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1215/2430] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:57 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:57.689726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.699877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c550d-1fdc-4c10-80f9-b5e94ecf8ae8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.704453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.705046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.705325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.705598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.706234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.707870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.718317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1216/2431] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:57.722142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.732871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d8957-1e32-4d31-b2d9-fa5eaf5502f2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.737717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dd0c7-1baf-4bc0-80bc-1625d1ef9822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.738314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd0c7-1baf-4bc0-80bc-1625d1ef9822 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.738604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd0c7-1baf-4bc0-80bc-1625d1ef9822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.738881 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd0c7-1baf-4bc0-80bc-1625d1ef9822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.739514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dd0c7-1baf-4bc0-80bc-1625d1ef9822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.741155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dd0c7-1baf-4bc0-80bc-1625d1ef9822 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.751791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1216/2432] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:57.755780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.765568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b814c-3ba5-4e5f-8662-33b4ae2e98db admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.770223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.771124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.771404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.771678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] PATH_INFO: `/v3/projects/fecbc7e0c1854fff88510818e7d2abf1/users/545a179ac9e7496d9d0b1879f7fbad95/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.786369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecbc7e0c1854fff88510818e7d2abf1, user_id=545a179ac9e7496d9d0b1879f7fbad95, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.789160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.802054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1217/2433] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:57 2022] PUT /identity/v3/projects/fecbc7e0c1854fff88510818e7d2abf1/users/545a179ac9e7496d9d0b1879f7fbad95/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:57.807800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ca9b1-63b3-4322-8e5d-1fa652a41b95 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.808201 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ca9b1-63b3-4322-8e5d-1fa652a41b95 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.808509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ca9b1-63b3-4322-8e5d-1fa652a41b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.828936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-655ca9b1-63b3-4322-8e5d-1fa652a41b95 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:57.830563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-655ca9b1-63b3-4322-8e5d-1fa652a41b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:57.891736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1217/2434] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:57.901618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:57.903326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:57.960089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a047140-de15-4a0e-8cf7-d99ee1a2c3c2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:57.963923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:57.964565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:57.964848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:57.965123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:57.974113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:57.975548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:57.985663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1218/2435] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:06:58.706113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:58.706636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:58.706772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:58.735804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] MFA Rules not processed for user `545a179ac9e7496d9d0b1879f7fbad95`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:58.737090 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:06:58.790733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:58.809375 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1218/2436] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:06:58.813869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:58.824201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34770008-f4ae-4d82-b4b2-6a779a3bb5cd service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:58.828911 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:58.829509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:58.829968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:58.830252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:58.830885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:58.833118 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:58.861223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1219/2437] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:06:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:58.865263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:58.876952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470868d-d759-4ea6-b0e7-5efc82f685a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:58.933969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:58.934581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:58.934865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:58.935136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:58.935768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:58.937714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:58.959378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1219/2438] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:06:58 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:06:58.963941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:58.974184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699ce76-f742-49ea-aa1e-e8f71d05aed3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:58.979318 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:58.979915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:58.980196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:58.980513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:58.981178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:58.982778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:58.993670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1220/2439] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:58.997861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:59.007917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b186254-7b53-4267-a33b-8ee7d6ed6fc4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:59.012559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:59.013205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:59.013560 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:59.013762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] PATH_INFO: `/v3/projects/e7e7d5ad109f43b19fe1a1d759c7efaf/users/3c5c3c86bc7e47d09f0e3c6b05b94a46/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:59.029295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e7d5ad109f43b19fe1a1d759c7efaf, user_id=3c5c3c86bc7e47d09f0e3c6b05b94a46, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:59.031865 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:59.044616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1220/2440] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:58 2022] PUT /identity/v3/projects/e7e7d5ad109f43b19fe1a1d759c7efaf/users/3c5c3c86bc7e47d09f0e3c6b05b94a46/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:59.048633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:59.058570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fa747-7738-49f0-a8da-67c619fee199 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:59.081700 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:06:59.118113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e35bd3-b33b-4258-97a8-b22dcf2ad641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:59.118853 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e35bd3-b33b-4258-97a8-b22dcf2ad641 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:59.119160 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e35bd3-b33b-4258-97a8-b22dcf2ad641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:59.119449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e35bd3-b33b-4258-97a8-b22dcf2ad641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:59.120065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e35bd3-b33b-4258-97a8-b22dcf2ad641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:59.121687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e35bd3-b33b-4258-97a8-b22dcf2ad641 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:59.131560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1221/2441] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:06:59.135795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:06:59.146175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca146dc9-7406-4a94-b445-e3285baeff24 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:06:59.151232 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:06:59.151870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:59.152149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:59.152460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] PATH_INFO: `/v3/projects/e7e7d5ad109f43b19fe1a1d759c7efaf/users/3c5c3c86bc7e47d09f0e3c6b05b94a46/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:59.154321 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e7d5ad109f43b19fe1a1d759c7efaf, user_id=3c5c3c86bc7e47d09f0e3c6b05b94a46, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:06:59.157001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:06:59.167602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1221/2442] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:06:59 2022] PUT /identity/v3/projects/e7e7d5ad109f43b19fe1a1d759c7efaf/users/3c5c3c86bc7e47d09f0e3c6b05b94a46/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:06:59.910538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:06:59.911094 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:06:59.911094 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:06:59.945421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] MFA Rules not processed for user `3c5c3c86bc7e47d09f0e3c6b05b94a46`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:06:59.947022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:00.021992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1222/2443] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:07:00.035388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:00.037409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:00.131020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c59b9-d41c-4075-b8f4-595a89d4f123 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:00.134979 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038559bc-7003-4812-9b0a-386a0f786243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:00.135583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038559bc-7003-4812-9b0a-386a0f786243 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:00.135869 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038559bc-7003-4812-9b0a-386a0f786243 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:00.136141 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038559bc-7003-4812-9b0a-386a0f786243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:00.178497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:00.203669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038559bc-7003-4812-9b0a-386a0f786243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:00.205142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038559bc-7003-4812-9b0a-386a0f786243 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:00.215014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1222/2444] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:07:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:07:01.569155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:01.570545 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:01.583136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865ffdd-b0ad-456b-a49a-2e6e44d42951 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:01.586829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:01.587420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:01.587702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:01.587976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:01.596844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:01.598252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:01.607767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1223/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:07:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:07:10.210017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=10 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:11.603501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=57 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:14.417615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:14.418111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:14.418221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:14.448670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:07:14.449884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:14.512626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1223/2446] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:07:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:07:14.565966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:14.567413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:14.637891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dc6f8-9088-4f1b-8ee8-4999e4d77713 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:14.643156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:14.643752 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:14.644036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:14.644379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:14.653879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:14.655472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:14.663755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1224/2447] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 13:07:15.978949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:16.076801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5af50-5659-4815-9bd1-afff34c06c42 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:16.087914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:16.088884 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:16.089331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:16.089786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] PATH_INFO: `/v3/users/545a179ac9e7496d9d0b1879f7fbad95` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:16.091118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] RBAC: Authorizing `identity:delete_user(user_id=545a179ac9e7496d9d0b1879f7fbad95)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:16.102638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:16.169022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '545a179ac9e7496d9d0b1879f7fbad95'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.188266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '545a179ac9e7496d9d0b1879f7fbad95'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.206502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '545a179ac9e7496d9d0b1879f7fbad95'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.217786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '545a179ac9e7496d9d0b1879f7fbad95'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.220749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '545a179ac9e7496d9d0b1879f7fbad95'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.229991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1224/2448] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:07:15 2022] DELETE /identity/v3/users/545a179ac9e7496d9d0b1879f7fbad95 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:07:16.442262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:16.461100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58c5bd-7bb5-4189-87b5-b6ee85fa1f26 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:16.551178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:16.551940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:16.552303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:16.552645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] PATH_INFO: `/v3/projects/fecbc7e0c1854fff88510818e7d2abf1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:16.554114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fecbc7e0c1854fff88510818e7d2abf1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:16.557870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:16.622621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fecbc7e0c1854fff88510818e7d2abf1', 'request_id': 'req-cefac9cf-88af-468c-82a1-6d4310f13692'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.628890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fecbc7e0c1854fff88510818e7d2abf1', 'request_id': 'req-cefac9cf-88af-468c-82a1-6d4310f13692'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.636467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fecbc7e0c1854fff88510818e7d2abf1', 'request_id': 'req-cefac9cf-88af-468c-82a1-6d4310f13692'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.650530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] The token cache is being invalidate because project fecbc7e0c1854fff88510818e7d2abf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:07:16.651146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.652136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.653932 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1225/2449] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:07:16 2022] DELETE /identity/v3/projects/fecbc7e0c1854fff88510818e7d2abf1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:07:16.658786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:16.660294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:16.674972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f808-8b76-4de4-9f01-067f7c03beab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:16.684208 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:16.752949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:16.753509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:16.753787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:16.754063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] PATH_INFO: `/v3/users/3c5c3c86bc7e47d09f0e3c6b05b94a46` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:16.755221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5c3c86bc7e47d09f0e3c6b05b94a46)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:16.757204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:16.819458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5c3c86bc7e47d09f0e3c6b05b94a46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.836525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5c3c86bc7e47d09f0e3c6b05b94a46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.845483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5c3c86bc7e47d09f0e3c6b05b94a46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.857165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5c3c86bc7e47d09f0e3c6b05b94a46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.857957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-02503d0a-d0b9-415b-a071-ec7489d96f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5c3c86bc7e47d09f0e3c6b05b94a46'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:16.864411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1225/2450] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:07:16 2022] DELETE /identity/v3/users/3c5c3c86bc7e47d09f0e3c6b05b94a46 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:07:17.059398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.060944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:17.078609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefac9cf-88af-468c-82a1-6d4310f13692 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.103354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:17.155783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.156464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.156741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.157009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] PATH_INFO: `/v3/projects/e7e7d5ad109f43b19fe1a1d759c7efaf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.160147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e7d5ad109f43b19fe1a1d759c7efaf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.160619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.212500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e7d5ad109f43b19fe1a1d759c7efaf', 'request_id': 'req-98770e34-8437-4688-a7ce-c0b0067c6a2e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:17.220075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e7d5ad109f43b19fe1a1d759c7efaf', 'request_id': 'req-98770e34-8437-4688-a7ce-c0b0067c6a2e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:17.227758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e7d5ad109f43b19fe1a1d759c7efaf', 'request_id': 'req-98770e34-8437-4688-a7ce-c0b0067c6a2e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:17.247283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] The token cache is being invalidate because project e7e7d5ad109f43b19fe1a1d759c7efaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:07:17.248808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:17.249489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-98770e34-8437-4688-a7ce-c0b0067c6a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:07:17.251166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1226/2451] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:07:17 2022] DELETE /identity/v3/projects/e7e7d5ad109f43b19fe1a1d759c7efaf => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:07:17.256085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.256630 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.256716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.284445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:07:17.287270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:17.410844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.411124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.411399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.417102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1226/2452] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:07:17.439582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:07:17.441596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:17.453867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1227/2453] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:07:17.458291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.473954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc374a2a-ce0b-4c1a-b13d-8ad07b1f0f98 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.479273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.479988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.480307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.480565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.481209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:07:17.481512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.483107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.498638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1227/2454] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:07:17.503618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.515282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb2211-78de-4147-af35-f6a8110c4101 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.523365 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.525828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.525828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.525828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.527054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.530402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.596540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1228/2455] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:07:17 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:07:17.600749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.614289 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ae853-754f-4441-be50-114f5f274c4b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.628540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:17.683578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.684328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.684621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.684897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.685610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.687745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.714933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1228/2456] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:07:17 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:07:17.721447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.733357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabeefe4-cdd6-49ed-8714-1b4d1c0d09d3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.738137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.738750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.739039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.739299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.739946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.741651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.755808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1229/2457] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:07:17.760330 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.771340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc905b8-20e4-4a10-841f-8f4458250c57 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.776963 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47cc27-1c63-40e2-8480-eeac32290e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.777591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47cc27-1c63-40e2-8480-eeac32290e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.777880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47cc27-1c63-40e2-8480-eeac32290e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.778149 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47cc27-1c63-40e2-8480-eeac32290e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.778794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47cc27-1c63-40e2-8480-eeac32290e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.780440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47cc27-1c63-40e2-8480-eeac32290e79 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.797318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1229/2458] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:07:17.798833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:17.809746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63591f7d-5e51-48c2-95b9-85b150b11898 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:17.814785 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:17.815476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.815765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.816040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] PATH_INFO: `/v3/projects/685d9418679047a1adb2573613bf575f/users/853fdd832d2a430090fc4ef26185df01/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.833614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=685d9418679047a1adb2573613bf575f, user_id=853fdd832d2a430090fc4ef26185df01, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:17.837558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:17.924222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1230/2459] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:07:17 2022] PUT /identity/v3/projects/685d9418679047a1adb2573613bf575f/users/853fdd832d2a430090fc4ef26185df01/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:07:17.934828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cb2d-9227-4c16-804a-c08334a0b56e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:17.935203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cb2d-9227-4c16-804a-c08334a0b56e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:17.935478 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cb2d-9227-4c16-804a-c08334a0b56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:17.967214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12c0cb2d-9227-4c16-804a-c08334a0b56e None None] MFA Rules not processed for user `853fdd832d2a430090fc4ef26185df01`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:07:17.968506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12c0cb2d-9227-4c16-804a-c08334a0b56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:18.047378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1230/2460] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:07:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:07:18.059135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:18.060861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:18.138799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:18.146403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e89ed-820c-4720-86ad-7c70fc774e05 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:18.151576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:18.155403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:18.156155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:18.156548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:18.166851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:18.168455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:18.178584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1231/2461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:07:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:07:28.042607 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:28.174323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:32.057540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:32.058025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:32.058119 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:32.076975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:07:32.078172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:32.133552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1231/2462] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:07:32.145414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:32.147053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:32.216590 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee1804-91de-4b6e-a7aa-cd54e3bdf03d service glance] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:32.221559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c5eb7-63a1-4b96-b1ae-c85ed52b87e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:32.222186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c5eb7-63a1-4b96-b1ae-c85ed52b87e3 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:32.222477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c5eb7-63a1-4b96-b1ae-c85ed52b87e3 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:32.222755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c5eb7-63a1-4b96-b1ae-c85ed52b87e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:32.231543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c5eb7-63a1-4b96-b1ae-c85ed52b87e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:32.233117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c5eb7-63a1-4b96-b1ae-c85ed52b87e3 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:32.241170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1232/2463] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:07:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:07:42.126818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=25 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:42.237802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=32 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:07:53.531443 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:07:53.533041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:53.597026 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c3854-8e6a-42a9-9ee8-3e26b7f690e3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:07:53.600733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:07:53.601348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:07:53.601629 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:07:53.601900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:07:53.603905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:07:53.674800 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:07:53.676969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:07:53.687594 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1232/2464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:07:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:02.304025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:02.304599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:02.304599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:02.324873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:02.328461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:02.335771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1233/2465] 10.222.0.57 () {58 vars in 967 bytes} [Thu Sep 15 13:08:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:02.350109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:02.362123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b2abf-f13e-40f8-a652-fb500d5d9794 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:02.368861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:02.369589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:02.369972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:02.370355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:02.380378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:02.382203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:02.393018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1233/2466] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:08:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:08:04.415091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:04.425839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e66d3-bc30-4d2c-b382-561415e91db5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:04.429821 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:04.430546 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:04.430852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:04.431166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:04.432963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:04.509615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:04.511055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:04.524169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1234/2467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:07.651309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:07.652777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:07.789810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5c52-be72-450e-ac38-fc02381c541f service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:07.801013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:07.801831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:07.802174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:07.802452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:07.806716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:07.807550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:07.819842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bbd04-e7b6-44b4-a0dd-f2ea2a2c7c8f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:07.828871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:07.830475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7268f1e0-72c4-43da-a8f1-e5dfb9d66e05 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:07.835610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:07.836443 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:07.836864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:07.837134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] PATH_INFO: `/v3/users/853fdd832d2a430090fc4ef26185df01` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:07.838442 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853fdd832d2a430090fc4ef26185df01)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:07.840518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:07.842171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1234/2468] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:07.903687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853fdd832d2a430090fc4ef26185df01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:07.910669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853fdd832d2a430090fc4ef26185df01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:07.923089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853fdd832d2a430090fc4ef26185df01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:07.923807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853fdd832d2a430090fc4ef26185df01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:07.932537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853fdd832d2a430090fc4ef26185df01'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:07.938976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1235/2469] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:07 2022] DELETE /identity/v3/users/853fdd832d2a430090fc4ef26185df01 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:07.943514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:07.943807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:07.944083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:07.963785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:07.965037 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:07.991324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:08.027169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1235/2470] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:08.037030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.038474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:08.103514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5dc8b-012c-4260-9a36-fd54a010f3c3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.107488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b940843-24d1-4a47-a7f7-a4261578af8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.108098 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b940843-24d1-4a47-a7f7-a4261578af8d service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.108401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b940843-24d1-4a47-a7f7-a4261578af8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.108672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b940843-24d1-4a47-a7f7-a4261578af8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.118317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b940843-24d1-4a47-a7f7-a4261578af8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.119765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b940843-24d1-4a47-a7f7-a4261578af8d service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.130131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1236/2471] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:08.403059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.405410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:08.421265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ce69b-7615-458d-9850-a8ed2637ca3c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.426154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.426807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.427121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.427645 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] PATH_INFO: `/v3/projects/685d9418679047a1adb2573613bf575f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.428776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=685d9418679047a1adb2573613bf575f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.430656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.481891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685d9418679047a1adb2573613bf575f', 'request_id': 'req-23602cac-5332-4a2b-ac0f-fd6a6e670474'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:08.487886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685d9418679047a1adb2573613bf575f', 'request_id': 'req-23602cac-5332-4a2b-ac0f-fd6a6e670474'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:08.494378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685d9418679047a1adb2573613bf575f', 'request_id': 'req-23602cac-5332-4a2b-ac0f-fd6a6e670474'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:08.508677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] The token cache is being invalidate because project 685d9418679047a1adb2573613bf575f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:08.508971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:08.509601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-23602cac-5332-4a2b-ac0f-fd6a6e670474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:08.515423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1236/2472] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:08 2022] DELETE /identity/v3/projects/685d9418679047a1adb2573613bf575f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:08.516811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.517103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.517373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.526408 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:08.542047 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:08.544534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:08.607144 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1237/2473] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:08.614808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.615114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.615386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.639553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:08.640159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:08.648640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1237/2474] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:08.652808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.663952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6543f4f-5345-41f2-b148-73eee079dba1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.670068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.670919 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.671207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.671483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.672138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:08.672473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.674101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.687264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1238/2475] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:08.693152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.704333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23fbfa-9fd4-4e1d-b82d-d45f67620085 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.709387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.710003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.710283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.710551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.711347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.713284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.752840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1238/2476] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:08 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:08.757070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.770619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7dd24-f08f-4170-9368-f74c487d5f18 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.836020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.836791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.836954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.837232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.837879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.839808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.863452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1239/2477] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:08 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:08.867778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.878410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7f36-12bf-4fae-82a3-3801ce8283bc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.883144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.883779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.884065 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.884366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.885007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.886604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.898923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1239/2478] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:08.903745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.913895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f34df-c3dc-4487-81a0-49012680f1c6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.918586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d995b8-0cca-4d58-b9b2-9137c7b22467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.919859 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d995b8-0cca-4d58-b9b2-9137c7b22467 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.920250 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d995b8-0cca-4d58-b9b2-9137c7b22467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.920559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d995b8-0cca-4d58-b9b2-9137c7b22467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.921200 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d995b8-0cca-4d58-b9b2-9137c7b22467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.922814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d995b8-0cca-4d58-b9b2-9137c7b22467 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.934053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1240/2479] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:08.938785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:08.949467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1a5f2-f4e1-4254-bf65-2800b59deffc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:08.954359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:08.955104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.955459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.955812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] PATH_INFO: `/v3/projects/8b815e753bbe4a54942550a564653fd8/users/fd09d576051f4b98a8a9b3ec0ca18bc8/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:08.972018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b815e753bbe4a54942550a564653fd8, user_id=fd09d576051f4b98a8a9b3ec0ca18bc8, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:08.974757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:08.990743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1240/2480] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:08 2022] PUT /identity/v3/projects/8b815e753bbe4a54942550a564653fd8/users/fd09d576051f4b98a8a9b3ec0ca18bc8/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:08.996147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:08.996627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:08.996962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:09.018103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:09.019253 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:09.088923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1241/2481] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:09.100550 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:09.102281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:09.163752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef5abb-0447-4210-a777-5e71185c0042 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:09.167403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:09.168028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:09.168339 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:09.168718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:09.175042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:09.179390 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:09.181284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:09.193230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1241/2482] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:11.410525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:11.411991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:11.426254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9704c-585e-4fb1-8b63-04e45e2ac37a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:11.437028 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:11.437701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:11.437986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:11.438263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] PATH_INFO: `/v3/users/381e3962c4e24a599e720303e9be17e2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:11.439579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381e3962c4e24a599e720303e9be17e2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:11.441655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:11.444492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:11.539791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381e3962c4e24a599e720303e9be17e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:11.549319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381e3962c4e24a599e720303e9be17e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:11.574295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381e3962c4e24a599e720303e9be17e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:11.575562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381e3962c4e24a599e720303e9be17e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:11.589924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381e3962c4e24a599e720303e9be17e2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:11.599170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1242/2483] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:11 2022] DELETE /identity/v3/users/381e3962c4e24a599e720303e9be17e2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:11.845640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:11.865378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e772aee-53a0-4c9d-a87a-7abf2feea428 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:11.965140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:11.966581 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:11.967096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:11.968830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] PATH_INFO: `/v3/projects/a6537f34327448c2b6acd5774dffc8c9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:11.971041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6537f34327448c2b6acd5774dffc8c9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:11.973929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:12.028520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6537f34327448c2b6acd5774dffc8c9', 'request_id': 'req-4f27395c-6fa6-460d-be54-5cea76d52219'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.035078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6537f34327448c2b6acd5774dffc8c9', 'request_id': 'req-4f27395c-6fa6-460d-be54-5cea76d52219'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.042241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6537f34327448c2b6acd5774dffc8c9', 'request_id': 'req-4f27395c-6fa6-460d-be54-5cea76d52219'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.061382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] The token cache is being invalidate because project a6537f34327448c2b6acd5774dffc8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:12.062008 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.062921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.065826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1242/2484] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:11 2022] DELETE /identity/v3/projects/a6537f34327448c2b6acd5774dffc8c9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:12.070199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:12.071557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:12.086224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8dc975-1f63-426f-a515-1c63e2fbb641 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:12.157276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:12.157942 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:12.158299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:12.158571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] PATH_INFO: `/v3/users/482042dc94d44d20b44c62c475236e08` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:12.159854 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=482042dc94d44d20b44c62c475236e08)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:12.161829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:12.220814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482042dc94d44d20b44c62c475236e08'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.228032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482042dc94d44d20b44c62c475236e08'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.241913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482042dc94d44d20b44c62c475236e08'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.242885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482042dc94d44d20b44c62c475236e08'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.251996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-593f797e-3520-4061-bdac-efda01975df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482042dc94d44d20b44c62c475236e08'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.254998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:12.260510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1243/2485] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:12 2022] DELETE /identity/v3/users/482042dc94d44d20b44c62c475236e08 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:12.508717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:12.510178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:12.525311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27395c-6fa6-460d-be54-5cea76d52219 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:12.599755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:12.601549 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:12.601915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:12.602401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] PATH_INFO: `/v3/projects/a6ba2d3a042b4248af4e59f196584800` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:12.604562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ba2d3a042b4248af4e59f196584800)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:12.607212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:12.651409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:12.680583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ba2d3a042b4248af4e59f196584800', 'request_id': 'req-2f1b4979-0477-48cc-9382-f7c6031ca46c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.688178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ba2d3a042b4248af4e59f196584800', 'request_id': 'req-2f1b4979-0477-48cc-9382-f7c6031ca46c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.694738 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ba2d3a042b4248af4e59f196584800', 'request_id': 'req-2f1b4979-0477-48cc-9382-f7c6031ca46c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.727934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] The token cache is being invalidate because project a6ba2d3a042b4248af4e59f196584800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:12.728238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.728926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:12.730789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1243/2486] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:12 2022] DELETE /identity/v3/projects/a6ba2d3a042b4248af4e59f196584800 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:14.131600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.132112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.132222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.161034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] MFA Rules not processed for user `fd09d576051f4b98a8a9b3ec0ca18bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:14.162158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:14.227469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1244/2487] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:14.232831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:14.234264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:14.305721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b4979-0477-48cc-9382-f7c6031ca46c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:14.310734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:14.311358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.311638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.311910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.312587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:14.314511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:14.343104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1244/2488] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:14 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:08:14.347391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:14.359510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761b259-4026-420e-8fcc-37ff1038ea85 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:14.435770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:14.436706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.437205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.437658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.443221 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:14.446271 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:14.471582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1245/2489] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:14 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:14.477023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:14.498595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042434f0-dc56-4523-b1df-6c6fa9a9d9bd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:14.503439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:14.503967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.504217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.504578 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.505174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:14.506793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:14.520703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1245/2490] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:14.523187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:14.533626 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0a09a-15e0-4329-a5c0-ccb359fad75d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:14.538683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:14.539356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.539637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.539974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] PATH_INFO: `/v3/projects/de7356a41c984673bc0591b18f223208/users/c5502530f4dc4fd8972321895a30f5f2/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.556412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7356a41c984673bc0591b18f223208, user_id=c5502530f4dc4fd8972321895a30f5f2, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:14.559372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:14.576087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1246/2491] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:14 2022] PUT /identity/v3/projects/de7356a41c984673bc0591b18f223208/users/c5502530f4dc4fd8972321895a30f5f2/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:14.580084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:14.593499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aaf57-604e-476c-8908-bbc4e7d2d687 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:14.661508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324d36f-62d1-4b86-bd22-f62787555497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:14.662342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d36f-62d1-4b86-bd22-f62787555497 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.662626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d36f-62d1-4b86-bd22-f62787555497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.662897 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d36f-62d1-4b86-bd22-f62787555497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.663535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324d36f-62d1-4b86-bd22-f62787555497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:14.665204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324d36f-62d1-4b86-bd22-f62787555497 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:14.676017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1246/2492] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:14.680479 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:14.692226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a312a3b-adc1-4312-beec-4500f2c0be1a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:14.697256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:14.697926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:14.698202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:14.698468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] PATH_INFO: `/v3/projects/de7356a41c984673bc0591b18f223208/users/c5502530f4dc4fd8972321895a30f5f2/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:14.701185 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7356a41c984673bc0591b18f223208, user_id=c5502530f4dc4fd8972321895a30f5f2, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:14.704579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:14.717723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1247/2493] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:14 2022] PUT /identity/v3/projects/de7356a41c984673bc0591b18f223208/users/c5502530f4dc4fd8972321895a30f5f2/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:19.311035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06810b-8f02-4c04-a0f1-39d80c978605 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:19.311964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06810b-8f02-4c04-a0f1-39d80c978605 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:19.312242 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06810b-8f02-4c04-a0f1-39d80c978605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:19.343931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bd06810b-8f02-4c04-a0f1-39d80c978605 None None] MFA Rules not processed for user `c5502530f4dc4fd8972321895a30f5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:19.345159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd06810b-8f02-4c04-a0f1-39d80c978605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:19.358943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:19.439887 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1247/2494] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:19.452519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:19.454315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:19.503623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:19.517695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5d7b-59a7-43cf-a5b5-7726eb2c82a3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:19.521684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:19.522310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:19.522589 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:19.522858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:19.582726 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:19.584165 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:19.593494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1248/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:19.756051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:19.756363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:19.756635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:19.777135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:19.778923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:19.843136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1248/2496] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:08:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:19.866515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:19.868050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:19.964773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263d6b2-b250-4760-8baa-2b83c9c6b40c service glance] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:19.971672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189f851-912d-49b4-9ba6-a238fd1ea45d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:19.972387 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189f851-912d-49b4-9ba6-a238fd1ea45d service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:19.972702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189f851-912d-49b4-9ba6-a238fd1ea45d service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:19.972984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189f851-912d-49b4-9ba6-a238fd1ea45d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:19.983767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189f851-912d-49b4-9ba6-a238fd1ea45d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:19.985913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189f851-912d-49b4-9ba6-a238fd1ea45d service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:19.995934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1249/2497] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:08:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 13:08:27.665194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:27.666780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:27.733670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7632e97-6566-4ed0-b531-20afc83a6528 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:27.737466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:27.738161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:27.738509 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:27.738868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:27.740708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:27.819519 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:27.820801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:27.834456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1249/2498] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:08:29.992462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=61 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:30.127824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:30.128116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:30.128502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:30.149955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] MFA Rules not processed for user `fd09d576051f4b98a8a9b3ec0ca18bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:30.151174 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:30.162707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1250/2499] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:30.199172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:30.205811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:30.216354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c07c1-97c2-431e-84f4-09ce4ec1a21e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:30.220679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:30.221533 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:30.222049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:30.222396 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:30.224386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:30.319482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:30.321088 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:30.344244 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1250/2500] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:31.183463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:31.193989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db10c51-5d2b-4f25-ba8c-44146392e727 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:31.198454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:31.201392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:31.201871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:31.202295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:31.204256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:31.221315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:31.223278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:31.237672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1251/2501] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:31.822195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:31.823779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:31.840575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75cbb0-b505-479d-9fec-880bbfa86507 service glance] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:31.847404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:31.848534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:31.849311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:31.850523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] PATH_INFO: `/v3/users/cff1145566f1408a9a22da9088581024` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:31.866633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff1145566f1408a9a22da9088581024)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:31.868961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:31.941093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff1145566f1408a9a22da9088581024'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:31.957282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff1145566f1408a9a22da9088581024'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:31.965374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff1145566f1408a9a22da9088581024'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:31.973680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff1145566f1408a9a22da9088581024'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:31.974456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-aa393aa5-de66-425b-8f07-ec27b981101d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff1145566f1408a9a22da9088581024'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:31.981025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1251/2502] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:31 2022] DELETE /identity/v3/users/cff1145566f1408a9a22da9088581024 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:32.161644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.175354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ec7cc-afeb-4238-95a9-1200115d0cd9 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.231908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.232593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.232844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.233128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] PATH_INFO: `/v3/projects/27776e610cf84b49af0fec4b95aa31f3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.243461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27776e610cf84b49af0fec4b95aa31f3)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.245437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.294279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27776e610cf84b49af0fec4b95aa31f3', 'request_id': 'req-23758b0a-1f1b-47b8-ae39-25405319fd2b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:32.299709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27776e610cf84b49af0fec4b95aa31f3', 'request_id': 'req-23758b0a-1f1b-47b8-ae39-25405319fd2b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:32.306140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27776e610cf84b49af0fec4b95aa31f3', 'request_id': 'req-23758b0a-1f1b-47b8-ae39-25405319fd2b'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:32.319863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] The token cache is being invalidate because project 27776e610cf84b49af0fec4b95aa31f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:32.320141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:32.320864 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-23758b0a-1f1b-47b8-ae39-25405319fd2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:32.322606 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1252/2503] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:32 2022] DELETE /identity/v3/projects/27776e610cf84b49af0fec4b95aa31f3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:32.328434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.328804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.329830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.354316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:32.355617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:32.389793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:32.430751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1252/2504] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:32.439147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.439543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.439842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.471490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:32.473042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:32.482440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1253/2505] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:32.486753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.496969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c7d63-01ca-4f7b-b2c9-1c86a448a038 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.502788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.502945 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.503225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.503505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.504150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:32.504493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.506126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.522856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1253/2506] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:32.527369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.538735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3ca50-dcb6-4481-a9fc-bb4c4db9e404 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.543710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.544356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.544642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.545128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.545766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.547866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.588870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1254/2507] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:32.593342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.603392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d34a78-6a79-4252-bae6-330b5393c254 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.666551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.667167 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.667451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.667727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.668677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.670626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.691906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1254/2508] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:32.696334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.708081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506e7da-ce4f-4f3b-9c81-9d923185af0d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.713875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.714487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.715317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.715625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.716298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.717960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.728608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1255/2509] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:08:32.732655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.743810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcaf8f3-88df-46a8-813f-c5d5ac3a7ebc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.749553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfd451-9c2e-42ac-a004-3ddedc3e981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.750132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd451-9c2e-42ac-a004-3ddedc3e981f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.750410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd451-9c2e-42ac-a004-3ddedc3e981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.750681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd451-9c2e-42ac-a004-3ddedc3e981f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.751364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd451-9c2e-42ac-a004-3ddedc3e981f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.753566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd451-9c2e-42ac-a004-3ddedc3e981f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.770075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1255/2510] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:08:32.774220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.784904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54f21d-0524-4fe0-ae92-7746eedd50a6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:32.789587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.789877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.790152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.794643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15239ed7-691f-46b8-8443-f5c833d840bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:32.795367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239ed7-691f-46b8-8443-f5c833d840bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.795669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239ed7-691f-46b8-8443-f5c833d840bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.795948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239ed7-691f-46b8-8443-f5c833d840bc admin admin] PATH_INFO: `/v3/projects/960a07eb537f4d06a0aa0a8c7ca33166/users/068569e5e2d943e7bdee58fdb4843625/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.810286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:32.814518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15239ed7-691f-46b8-8443-f5c833d840bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=960a07eb537f4d06a0aa0a8c7ca33166, user_id=068569e5e2d943e7bdee58fdb4843625, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:32.815507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:32.816877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:32.818057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15239ed7-691f-46b8-8443-f5c833d840bc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:32.835452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1256/2511] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:32 2022] PUT /identity/v3/projects/960a07eb537f4d06a0aa0a8c7ca33166/users/068569e5e2d943e7bdee58fdb4843625/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:32.847869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:32.848169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:32.848476 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:32.881294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] MFA Rules not processed for user `068569e5e2d943e7bdee58fdb4843625`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:32.882435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:32.897451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1256/2512] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:32.914508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.916003 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:32.957345 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1257/2513] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:32.961577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:32.969096 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:33.005770 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc4f55-7abd-45a1-bfc0-e8bb2e31a3e6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.040897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f603ae6-0683-4c9b-8c3f-a6c2a16604b1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.046092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.047244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.047551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.047831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.048652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.050454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.076076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.076871 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.077160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.077515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.090611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1258/2514] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:32 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:33.097297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:33.161472 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.163138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.173873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1257/2515] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:08:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:08:33.184583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a993e8f-06c4-48a1-88c5-b97e75760829 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.189742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f836e-474f-4392-a698-92302b7747da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.190537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f836e-474f-4392-a698-92302b7747da admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.190829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f836e-474f-4392-a698-92302b7747da admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.191108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f836e-474f-4392-a698-92302b7747da admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.192011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f836e-474f-4392-a698-92302b7747da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.194949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f836e-474f-4392-a698-92302b7747da admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.211657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:33.242177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1259/2516] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:33 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:33.245972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:33.259784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b8d73-6373-414d-8b10-501c2a371d8d service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.266557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.267408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.270703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.271177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.272385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.274263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.288058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1258/2517] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:08:33.292678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f836e-474f-4392-a698-92302b7747da admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:33.306961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f836e-474f-4392-a698-92302b7747da admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.319637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.320569 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.321708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.322534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] PATH_INFO: `/v3/projects/ebffc90b5db847de85d333bb0419bc61/users/9d62cd0bd962464abaf2cc58d584861d/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.368870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebffc90b5db847de85d333bb0419bc61, user_id=9d62cd0bd962464abaf2cc58d584861d, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.373395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.411470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1260/2518] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:33 2022] PUT /identity/v3/projects/ebffc90b5db847de85d333bb0419bc61/users/9d62cd0bd962464abaf2cc58d584861d/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:33.412976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:33.445731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a0801-f307-4fd6-ab5a-eb495d7f19ac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.578057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c4913-ad90-4bec-80e7-fccb6e07d8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.578690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c4913-ad90-4bec-80e7-fccb6e07d8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.578978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c4913-ad90-4bec-80e7-fccb6e07d8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.579310 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c4913-ad90-4bec-80e7-fccb6e07d8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.579955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c4913-ad90-4bec-80e7-fccb6e07d8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.581997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c4913-ad90-4bec-80e7-fccb6e07d8fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.599668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1259/2519] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:33.600231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:33.625599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ab71b-06bc-45a1-9633-fe251fbc6749 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.640992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.642681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.642983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.643258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] PATH_INFO: `/v3/projects/ebffc90b5db847de85d333bb0419bc61/users/9d62cd0bd962464abaf2cc58d584861d/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.649108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebffc90b5db847de85d333bb0419bc61, user_id=9d62cd0bd962464abaf2cc58d584861d, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:33.651925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:33.678917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1261/2520] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:33 2022] PUT /identity/v3/projects/ebffc90b5db847de85d333bb0419bc61/users/9d62cd0bd962464abaf2cc58d584861d/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:33.694438 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.694758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.695010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:33.746317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:33.766408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] MFA Rules not processed for user `9d62cd0bd962464abaf2cc58d584861d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:33.767669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:33.892721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1260/2521] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:33.906761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:33.908564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:33.992862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40367a-8f9a-4e79-a91b-3030fc9065ab admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:33.997473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ae0d6-f567-46c0-98cd-3f7e9ddaec6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:33.998102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ae0d6-f567-46c0-98cd-3f7e9ddaec6e service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:33.998380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ae0d6-f567-46c0-98cd-3f7e9ddaec6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:33.998652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ae0d6-f567-46c0-98cd-3f7e9ddaec6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:34.011491 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ae0d6-f567-46c0-98cd-3f7e9ddaec6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:34.018012 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ae0d6-f567-46c0-98cd-3f7e9ddaec6e service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:34.032372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1262/2522] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:34.589627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:34.661223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bfa73-0b93-4d3c-9c7c-365de98b2864 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:34.665762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:34.666400 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:34.666679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:34.666951 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] PATH_INFO: `/v3/users/068569e5e2d943e7bdee58fdb4843625` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:34.668873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] RBAC: Authorizing `identity:delete_user(user_id=068569e5e2d943e7bdee58fdb4843625)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:34.670958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:34.736194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068569e5e2d943e7bdee58fdb4843625'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:34.750091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068569e5e2d943e7bdee58fdb4843625'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:34.763445 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068569e5e2d943e7bdee58fdb4843625'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:34.773291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068569e5e2d943e7bdee58fdb4843625'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:34.773996 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068569e5e2d943e7bdee58fdb4843625'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:34.780768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1261/2523] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:34 2022] DELETE /identity/v3/users/068569e5e2d943e7bdee58fdb4843625 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:34.785093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:34.785610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:34.785707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:34.813180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:34.814455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:34.906579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1263/2524] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:34.917235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:34.919146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:34.943923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:34.994346 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2c0a6-7f67-4535-8fad-f38faa3f810e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:34.998624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:34.999323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:34.999685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:35.000024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:35.010077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:35.011640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:35.025231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1262/2525] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:35.341215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:35.342949 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:35.363193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15577c58-5eef-4a38-8022-d4e109944bed None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:35.368478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:35.369022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:35.369300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:35.369719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] PATH_INFO: `/v3/projects/960a07eb537f4d06a0aa0a8c7ca33166` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:35.371074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=960a07eb537f4d06a0aa0a8c7ca33166)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:35.373210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:35.423758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960a07eb537f4d06a0aa0a8c7ca33166', 'request_id': 'req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.429008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:35.430834 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960a07eb537f4d06a0aa0a8c7ca33166', 'request_id': 'req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.441721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960a07eb537f4d06a0aa0a8c7ca33166', 'request_id': 'req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.455736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] The token cache is being invalidate because project 960a07eb537f4d06a0aa0a8c7ca33166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:35.456079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.460676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.462461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1264/2526] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:35 2022] DELETE /identity/v3/projects/960a07eb537f4d06a0aa0a8c7ca33166 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:35.466459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:35.477100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4fa34-6c1d-4ef0-8fc7-8e7f8c90cf36 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:35.542491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:35.543719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:35.544101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:35.544477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] PATH_INFO: `/v3/users/9d62cd0bd962464abaf2cc58d584861d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:35.545712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d62cd0bd962464abaf2cc58d584861d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:35.547638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:35.605858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d62cd0bd962464abaf2cc58d584861d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.618408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d62cd0bd962464abaf2cc58d584861d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.626307 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d62cd0bd962464abaf2cc58d584861d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.634083 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d62cd0bd962464abaf2cc58d584861d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.634716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2ee3055a-77e2-4e5c-b615-fd3d8ad037ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d62cd0bd962464abaf2cc58d584861d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:35.642542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1263/2527] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:35 2022] DELETE /identity/v3/users/9d62cd0bd962464abaf2cc58d584861d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:35.946369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:35.948092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:35.976970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c9ce7-21c1-4cfb-aec2-63c791aaeef0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.059539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.060102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.060414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.060693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] PATH_INFO: `/v3/projects/ebffc90b5db847de85d333bb0419bc61` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.061826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebffc90b5db847de85d333bb0419bc61)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.063725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.127407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebffc90b5db847de85d333bb0419bc61', 'request_id': 'req-52aa1b98-7591-4db1-aa04-6854d697b1a2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:36.133411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebffc90b5db847de85d333bb0419bc61', 'request_id': 'req-52aa1b98-7591-4db1-aa04-6854d697b1a2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:36.149412 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebffc90b5db847de85d333bb0419bc61', 'request_id': 'req-52aa1b98-7591-4db1-aa04-6854d697b1a2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:36.164910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] The token cache is being invalidate because project ebffc90b5db847de85d333bb0419bc61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:36.165351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:36.166223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52aa1b98-7591-4db1-aa04-6854d697b1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:36.168307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1265/2528] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:35 2022] DELETE /identity/v3/projects/ebffc90b5db847de85d333bb0419bc61 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:36.173989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832aed1-e845-42e1-982e-327026296788 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.173989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832aed1-e845-42e1-982e-327026296788 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.173989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832aed1-e845-42e1-982e-327026296788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.216765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2832aed1-e845-42e1-982e-327026296788 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:36.218012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2832aed1-e845-42e1-982e-327026296788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:36.260399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:36.309795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1264/2529] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:36.319130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.319420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.319705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.341941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:36.343169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:36.351607 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1266/2530] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:36.356110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832aed1-e845-42e1-982e-327026296788 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.366563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832aed1-e845-42e1-982e-327026296788 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.371985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.372712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.373061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.373397 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.374115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:36.374490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.376444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.392093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1265/2531] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:36.396779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.408159 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41950961-e09f-404e-ab2b-0a253b15f026 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.413742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.414430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.414781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.415125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.415833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.417995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.448095 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1267/2532] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:36.453079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.467136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9528e6-2066-43b3-afb5-1f288a630e1a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.536826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.537461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.537745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.538020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.538646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.540634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.567136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1266/2533] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:08:36.571086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.582962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1eef4-b5a2-435b-a515-1a7418d0cd19 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.587895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.588552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.588839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.589105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.590690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.592335 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.603053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1268/2534] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:36.608822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.619218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07095739-670b-438d-b772-3aac48b6c3ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.625450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9e3ad-636c-4b54-aec6-1a6224c01bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.626139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e3ad-636c-4b54-aec6-1a6224c01bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.626485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e3ad-636c-4b54-aec6-1a6224c01bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.626821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9e3ad-636c-4b54-aec6-1a6224c01bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.627542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9e3ad-636c-4b54-aec6-1a6224c01bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.629240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9e3ad-636c-4b54-aec6-1a6224c01bd4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.644538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1267/2535] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:36.645666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.657530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69446d11-303d-48c9-8061-5528dcdc8b9d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.662914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.663931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.664222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.664537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] PATH_INFO: `/v3/projects/838b6ec2f3a94508a06a493949f3d042/users/e97aa973ceee459a963b6b2fa2616523/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.681440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] RBAC: Authorizing `identity:create_grant(project_id=838b6ec2f3a94508a06a493949f3d042, user_id=e97aa973ceee459a963b6b2fa2616523, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.684207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.698690 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1269/2536] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:36 2022] PUT /identity/v3/projects/838b6ec2f3a94508a06a493949f3d042/users/e97aa973ceee459a963b6b2fa2616523/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:36.709818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.710109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.710383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.746181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] MFA Rules not processed for user `e97aa973ceee459a963b6b2fa2616523`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:36.747855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:36.817866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1268/2537] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:36.822488 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.833746 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:36.893894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5904cbd-d481-43ef-89ae-baa7a289612f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.898863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:36.899474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:36.899749 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:36.900020 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:36.900678 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:36.903342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:36.938505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1270/2538] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:36.939975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:36.953782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba600a8-aa6f-4775-9773-911cd4f57588 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:36.986395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:37.019483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.020092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.020420 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.020697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.021323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.023538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.051490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1269/2539] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:36 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:37.052968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.064890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249bf4-6380-4b4b-8f92-7010a40c0e60 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:37.069672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.070208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.070490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.070763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.071405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.073237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.085464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1271/2540] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:37.090291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.105364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bae877-670f-412f-928d-bcdd6d3fe2a3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:37.110504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.111171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.111449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.111720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] PATH_INFO: `/v3/projects/7ae2a0bae13c4187b9162987a37cfdcb/users/5949f128eb6143e5983b451df780e044/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.132316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae2a0bae13c4187b9162987a37cfdcb, user_id=5949f128eb6143e5983b451df780e044, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.135138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.150900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1270/2541] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:37 2022] PUT /identity/v3/projects/7ae2a0bae13c4187b9162987a37cfdcb/users/5949f128eb6143e5983b451df780e044/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:37.156063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.167664 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f34be-8d7a-48c3-82b1-adc6997ce96a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:37.228111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14e606-8b7b-4117-b800-c39e0cca568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.228841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e606-8b7b-4117-b800-c39e0cca568f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.229127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e606-8b7b-4117-b800-c39e0cca568f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.229401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e606-8b7b-4117-b800-c39e0cca568f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.230048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14e606-8b7b-4117-b800-c39e0cca568f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.232332 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14e606-8b7b-4117-b800-c39e0cca568f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.246985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1272/2542] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:37.248169 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.258759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d824bc-744a-421a-9363-b784cdf8fd3b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:37.263850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.264575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.264863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.265164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] PATH_INFO: `/v3/projects/7ae2a0bae13c4187b9162987a37cfdcb/users/5949f128eb6143e5983b451df780e044/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.267856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae2a0bae13c4187b9162987a37cfdcb, user_id=5949f128eb6143e5983b451df780e044, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.270044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.282492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1271/2543] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:37 2022] PUT /identity/v3/projects/7ae2a0bae13c4187b9162987a37cfdcb/users/5949f128eb6143e5983b451df780e044/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:37.293826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.294295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.294427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.332961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] MFA Rules not processed for user `5949f128eb6143e5983b451df780e044`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:37.334522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:37.443232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1273/2544] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:37.466042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.467901 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:37.545437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744eea9-b370-4ee4-8ffe-f6b2850f7e08 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:37.549640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044aefab-312e-4d71-afa2-adf0694d58c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.550267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aefab-312e-4d71-afa2-adf0694d58c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.550547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aefab-312e-4d71-afa2-adf0694d58c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.550819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044aefab-312e-4d71-afa2-adf0694d58c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.561840 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aefab-312e-4d71-afa2-adf0694d58c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.563372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044aefab-312e-4d71-afa2-adf0694d58c4 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.573548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1272/2545] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:37.716597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.721867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:37.786470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9b73f-5519-49e5-b7e9-b5bc352ea800 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:37.791695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:37.792631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.794074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.794376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] PATH_INFO: `/v3/users/e97aa973ceee459a963b6b2fa2616523` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.795364 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97aa973ceee459a963b6b2fa2616523)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:37.797328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:37.850622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97aa973ceee459a963b6b2fa2616523'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:37.858228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97aa973ceee459a963b6b2fa2616523'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:37.870322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97aa973ceee459a963b6b2fa2616523'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:37.871601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97aa973ceee459a963b6b2fa2616523'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:37.880019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97aa973ceee459a963b6b2fa2616523'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:37.890687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1274/2546] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:37 2022] DELETE /identity/v3/users/e97aa973ceee459a963b6b2fa2616523 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:37.892679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:37.893182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:37.893291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:37.913434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:37.914665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:37.979044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1273/2547] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:37.987802 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:37.989323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:38.053128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f77103-f25f-44a1-8c04-8066b077afbd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:38.056956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:38.057635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:38.057884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:38.058176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:38.068768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:38.070247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:38.080929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1275/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:38.376968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:38.378428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:38.401333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fee3ea-8c8a-487f-99c0-e32431e88587 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:38.408382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:38.409081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:38.409306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:38.409585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] PATH_INFO: `/v3/projects/838b6ec2f3a94508a06a493949f3d042` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:38.410710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838b6ec2f3a94508a06a493949f3d042)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:38.413347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:38.443217 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:38.470421 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838b6ec2f3a94508a06a493949f3d042', 'request_id': 'req-d9d43160-244b-4801-a8a0-71e64433e987'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.481211 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838b6ec2f3a94508a06a493949f3d042', 'request_id': 'req-d9d43160-244b-4801-a8a0-71e64433e987'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.488007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838b6ec2f3a94508a06a493949f3d042', 'request_id': 'req-d9d43160-244b-4801-a8a0-71e64433e987'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.509755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] The token cache is being invalidate because project 838b6ec2f3a94508a06a493949f3d042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:38.510150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.510790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.513001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1274/2549] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:38 2022] DELETE /identity/v3/projects/838b6ec2f3a94508a06a493949f3d042 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:38.517052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:38.527261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bac39-5fdd-4b9e-9cfe-14d0b1790a16 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:38.555114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:38.605992 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:38.606702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:38.607093 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:38.607375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] PATH_INFO: `/v3/users/5949f128eb6143e5983b451df780e044` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:38.608618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5949f128eb6143e5983b451df780e044)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:38.613426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:38.675424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5949f128eb6143e5983b451df780e044'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.682699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5949f128eb6143e5983b451df780e044'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.700123 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5949f128eb6143e5983b451df780e044'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.701112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5949f128eb6143e5983b451df780e044'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.716056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4510324d-5de7-4057-a1e7-eefd07cda85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5949f128eb6143e5983b451df780e044'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:38.723759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1276/2550] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:38 2022] DELETE /identity/v3/users/5949f128eb6143e5983b451df780e044 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:39.037500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.039006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:39.075041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d43160-244b-4801-a8a0-71e64433e987 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.147568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.148212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.148531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.148803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] PATH_INFO: `/v3/projects/7ae2a0bae13c4187b9162987a37cfdcb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.150099 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae2a0bae13c4187b9162987a37cfdcb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.152041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.200965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae2a0bae13c4187b9162987a37cfdcb', 'request_id': 'req-0d2fa46e-bbd7-4478-b793-4ab5379109bf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:39.209986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae2a0bae13c4187b9162987a37cfdcb', 'request_id': 'req-0d2fa46e-bbd7-4478-b793-4ab5379109bf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:39.217688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae2a0bae13c4187b9162987a37cfdcb', 'request_id': 'req-0d2fa46e-bbd7-4478-b793-4ab5379109bf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:39.231538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] The token cache is being invalidate because project 7ae2a0bae13c4187b9162987a37cfdcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:39.231824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:39.232553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0d2fa46e-bbd7-4478-b793-4ab5379109bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:39.234353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1275/2551] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:39 2022] DELETE /identity/v3/projects/7ae2a0bae13c4187b9162987a37cfdcb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:39.239453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.239934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.240086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.263054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:39.264290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:39.326599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1277/2552] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:39.335813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.336097 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.336409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.358082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:39.359287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:39.367933 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1276/2553] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:39.373030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.383163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c3ea3-3808-41d9-aea3-6b653711979d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.387927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.388575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.388858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.389133 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.389789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:39.390089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.393418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.405436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1278/2554] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:08:39.409766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.420755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3be9ec-ddd6-4a3a-9c48-74027ee8a59b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.425736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.426351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.426633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.426904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.427535 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.429503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.461573 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1277/2555] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:39 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:39.463214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.474045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e119f-2a3f-4a05-a1dc-c0e9026ef460 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.507156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:39.541805 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.542418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.542793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.543604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.544348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.546262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.567389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1279/2556] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:39 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:39.571543 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.584144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06d837-b567-4778-956b-c8d87c04e1e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.593055 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.593870 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.594939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.595404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.596049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.597741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.609934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1278/2557] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:39.614038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.624612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb37bd4-15d5-4e3e-a36c-fc34155d2192 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.629278 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4a5df-5ded-43fe-bc1d-eedde3eb76bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.629909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a5df-5ded-43fe-bc1d-eedde3eb76bf admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.630192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a5df-5ded-43fe-bc1d-eedde3eb76bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.630467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4a5df-5ded-43fe-bc1d-eedde3eb76bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.631112 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4a5df-5ded-43fe-bc1d-eedde3eb76bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.632977 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4a5df-5ded-43fe-bc1d-eedde3eb76bf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.643665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1280/2558] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:39.648758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.655302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:39.660727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08f9a8-004b-4589-b19c-e00209aa95b6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.668492 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.669485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.669768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.670431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] PATH_INFO: `/v3/projects/774ce43ebb094ac082c95a2180a21bbf/users/1922702da1e440239a3b770affdffdbf/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.685727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] RBAC: Authorizing `identity:create_grant(project_id=774ce43ebb094ac082c95a2180a21bbf, user_id=1922702da1e440239a3b770affdffdbf, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.688422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.703078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1279/2559] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:39 2022] PUT /identity/v3/projects/774ce43ebb094ac082c95a2180a21bbf/users/1922702da1e440239a3b770affdffdbf/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:39.713536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.713830 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.714036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.747852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] MFA Rules not processed for user `1922702da1e440239a3b770affdffdbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:39.749222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:39.822508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1281/2560] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:39.835257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:39.837311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:39.916427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25e648-a4c0-495b-ab38-bed8fecb039a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:39.920704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c519f-8b88-449f-b8e9-b8e8e98fe99f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:39.921325 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c519f-8b88-449f-b8e9-b8e8e98fe99f service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:39.921604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c519f-8b88-449f-b8e9-b8e8e98fe99f service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:39.921879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c519f-8b88-449f-b8e9-b8e8e98fe99f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:39.932928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c519f-8b88-449f-b8e9-b8e8e98fe99f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:39.934638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c519f-8b88-449f-b8e9-b8e8e98fe99f service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:39.945273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1280/2561] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:39.969404 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:40.041560 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a71b8f-7356-4620-bf77-5eeb34b0acf4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:40.047906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:40.048592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:40.048950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:40.049128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] PATH_INFO: `/v3/users/1922702da1e440239a3b770affdffdbf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:40.050268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1922702da1e440239a3b770affdffdbf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:40.052191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:40.124565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:40.127907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1922702da1e440239a3b770affdffdbf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.138893 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1922702da1e440239a3b770affdffdbf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.154724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1922702da1e440239a3b770affdffdbf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.155563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1922702da1e440239a3b770affdffdbf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.166187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1922702da1e440239a3b770affdffdbf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.173428 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1282/2562] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:39 2022] DELETE /identity/v3/users/1922702da1e440239a3b770affdffdbf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:40.178007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:40.178466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:40.178667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:40.205867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:40.207600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:40.297670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1281/2563] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:40.310241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:40.312464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:40.422603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1eebf-1abe-4c5c-8c18-52c801feb72a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:40.426694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cd499-0dc4-4719-bad5-ad4ccd31bbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:40.427671 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cd499-0dc4-4719-bad5-ad4ccd31bbaf service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:40.428025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cd499-0dc4-4719-bad5-ad4ccd31bbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:40.428399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cd499-0dc4-4719-bad5-ad4ccd31bbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:40.439204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cd499-0dc4-4719-bad5-ad4ccd31bbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:40.440808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905cd499-0dc4-4719-bad5-ad4ccd31bbaf service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:40.452415 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1283/2564] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:40.753580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:40.755056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:40.781252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d578faf-eff9-48c0-9487-714ba38c69da None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:40.788461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:40.789123 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:40.789442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:40.789655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] PATH_INFO: `/v3/projects/774ce43ebb094ac082c95a2180a21bbf` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:40.790784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] RBAC: Authorizing `identity:delete_project(project_id=774ce43ebb094ac082c95a2180a21bbf)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:40.793762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:40.846939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774ce43ebb094ac082c95a2180a21bbf', 'request_id': 'req-063dc047-f95b-46a6-bab2-836a04a2694d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.853228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774ce43ebb094ac082c95a2180a21bbf', 'request_id': 'req-063dc047-f95b-46a6-bab2-836a04a2694d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.860817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774ce43ebb094ac082c95a2180a21bbf', 'request_id': 'req-063dc047-f95b-46a6-bab2-836a04a2694d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.876923 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] The token cache is being invalidate because project 774ce43ebb094ac082c95a2180a21bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:40.877220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.877875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-063dc047-f95b-46a6-bab2-836a04a2694d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:40.879376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1282/2565] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:40 2022] DELETE /identity/v3/projects/774ce43ebb094ac082c95a2180a21bbf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:08:40.884294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:40.884596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:40.884877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:40.906451 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:40.907680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:40.974740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1284/2566] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:40.981114 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:40.981610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:40.982017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:40.990678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:41.005711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:41.007036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:41.023286 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1283/2567] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:41.028823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.039691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525af991-3bb4-4647-a92f-f2c049767709 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.045782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.046403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.046685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.046956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.047603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:41.047907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.050625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.064832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1285/2568] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:41.069355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.081315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5681e4-4fe2-4500-bee1-bc61a8e191a9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.089275 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.090507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.090726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.090995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.091685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.093665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.128481 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1284/2569] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:41 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:08:41.132067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.143453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa1add-c750-41d9-9001-b727680ed88f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.196779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:41.208234 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.208881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.209164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.209440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.210077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.211998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.236311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1286/2570] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:41 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:08:41.240235 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.251230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b3c73-e207-4df1-9189-6e9ae7debf92 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.256172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.256825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.257106 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.257376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.258040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.259715 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.274233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1285/2571] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:08:41.279229 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.289241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab87e93-e760-43e4-bacb-a1a3c7767f9d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.294223 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfe053-f31e-427c-89a8-8d37e66640a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.294827 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfe053-f31e-427c-89a8-8d37e66640a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.295106 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfe053-f31e-427c-89a8-8d37e66640a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.295383 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfe053-f31e-427c-89a8-8d37e66640a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.296021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfe053-f31e-427c-89a8-8d37e66640a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.297720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfe053-f31e-427c-89a8-8d37e66640a5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.309290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1287/2572] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:41.313685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.333655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4548127-30f5-41ed-96ac-0bfa716779bd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.342122 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.342709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.343143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.343579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] PATH_INFO: `/v3/projects/9c22a7ea3edb495abe25149184dd876d/users/fb80dccf80584a1ca6d3cd5218c012bf/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.367268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c22a7ea3edb495abe25149184dd876d, user_id=fb80dccf80584a1ca6d3cd5218c012bf, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.370422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.392967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1286/2573] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:41 2022] PUT /identity/v3/projects/9c22a7ea3edb495abe25149184dd876d/users/fb80dccf80584a1ca6d3cd5218c012bf/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:41.405604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.406179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.406341 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.438687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] MFA Rules not processed for user `fb80dccf80584a1ca6d3cd5218c012bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:41.440724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:41.526540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1288/2574] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:41.539464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:41.541682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:41.617101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dbf8f-05be-4232-8ecf-faf9a980cbcb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:41.621576 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0313ba-f231-4a86-a308-570092a80431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:41.622198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0313ba-f231-4a86-a308-570092a80431 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:41.622479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0313ba-f231-4a86-a308-570092a80431 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:41.622759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0313ba-f231-4a86-a308-570092a80431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:41.631720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0313ba-f231-4a86-a308-570092a80431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:41.633193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0313ba-f231-4a86-a308-570092a80431 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:41.643389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1287/2575] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:08:51.518075 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:51.638835 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:52.164079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:52.224840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a75821-ef1c-4c96-b5a7-406ad3490469 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:52.229283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:52.229906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:52.230187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:52.230461 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] PATH_INFO: `/v3/users/fb80dccf80584a1ca6d3cd5218c012bf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:52.231568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb80dccf80584a1ca6d3cd5218c012bf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:52.233503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:52.280909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb80dccf80584a1ca6d3cd5218c012bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.287661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb80dccf80584a1ca6d3cd5218c012bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.299447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb80dccf80584a1ca6d3cd5218c012bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.300104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb80dccf80584a1ca6d3cd5218c012bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.307308 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb80dccf80584a1ca6d3cd5218c012bf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.313918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1289/2576] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:52 2022] DELETE /identity/v3/users/fb80dccf80584a1ca6d3cd5218c012bf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:52.318457 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:52.318748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:52.319023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:52.338038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:52.339257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:52.397973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1288/2577] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:52.407781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:52.409193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:52.467151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269f613-e6d9-48fc-b521-5185d0732ce5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:52.470777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98c83a-914e-4c2b-a90e-3bf06e067394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:52.471377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c83a-914e-4c2b-a90e-3bf06e067394 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:52.471658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c83a-914e-4c2b-a90e-3bf06e067394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:52.471927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98c83a-914e-4c2b-a90e-3bf06e067394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:52.480890 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98c83a-914e-4c2b-a90e-3bf06e067394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:52.482964 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98c83a-914e-4c2b-a90e-3bf06e067394 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:52.492178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1290/2578] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:52.760868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:52.762630 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:52.779648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44403a-053f-4da0-8adb-b944a4f8a041 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:52.784937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:52.785585 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:52.785868 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:52.786165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] PATH_INFO: `/v3/projects/9c22a7ea3edb495abe25149184dd876d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:52.787297 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c22a7ea3edb495abe25149184dd876d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:52.789469 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:52.843736 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c22a7ea3edb495abe25149184dd876d', 'request_id': 'req-3c370e52-7812-47c8-93c9-3798510edec4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.849592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c22a7ea3edb495abe25149184dd876d', 'request_id': 'req-3c370e52-7812-47c8-93c9-3798510edec4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.855858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c22a7ea3edb495abe25149184dd876d', 'request_id': 'req-3c370e52-7812-47c8-93c9-3798510edec4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.869145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] The token cache is being invalidate because project 9c22a7ea3edb495abe25149184dd876d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:52.869432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.870040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3c370e52-7812-47c8-93c9-3798510edec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:52.871500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1289/2579] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:52 2022] DELETE /identity/v3/projects/9c22a7ea3edb495abe25149184dd876d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:52.876999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:52.877470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:52.877565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:52.897806 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:52.899015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:52.915521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:52.959686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1291/2580] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:52.968337 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:52.968627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:52.968903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:52.990306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:52.991493 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:53.000062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1290/2581] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:53.004164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.014056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe9fd4-e692-481e-8e14-21d0eeab0898 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.019652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.020260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.020562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.020829 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.021790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:53.022178 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.023770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.037104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1292/2582] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:53.041527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.051965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b45f6b-f497-4833-8801-c334fd10624e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.056825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.057502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.057848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.058183 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.058874 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.061376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.094750 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1291/2583] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:53 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:53.098759 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.110887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db088c-c156-4a5d-a695-9c54727be2a5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.169385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.170252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.170554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.170822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.171447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.173381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.194358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1293/2584] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:53 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:53.198515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.209338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09814a7e-89cd-46ab-ab0d-1fbc300789e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.213994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.214527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.214807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.215081 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.215721 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.217379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.227151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1292/2585] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:53.231056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.241652 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4aae6-6776-423a-a68d-ebc426e987ee admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.246316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8cc87-33c3-4035-9f9b-8278a53a1e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.246901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8cc87-33c3-4035-9f9b-8278a53a1e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.247765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8cc87-33c3-4035-9f9b-8278a53a1e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.248310 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8cc87-33c3-4035-9f9b-8278a53a1e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.248984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8cc87-33c3-4035-9f9b-8278a53a1e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.250565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8cc87-33c3-4035-9f9b-8278a53a1e50 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.261078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1294/2586] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:53.265857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.276249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d7abe-1619-4b8e-8953-20fe6e54fbc0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.281496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.282143 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.282423 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.282703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] PATH_INFO: `/v3/projects/95bcf62929e24586a49f46020470c975/users/b69b177927e14f80ae61c18791b25d27/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.297931 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=95bcf62929e24586a49f46020470c975, user_id=b69b177927e14f80ae61c18791b25d27, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.300595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.313823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1293/2587] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:53 2022] PUT /identity/v3/projects/95bcf62929e24586a49f46020470c975/users/b69b177927e14f80ae61c18791b25d27/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:53.325036 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.325891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.326351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.353658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] MFA Rules not processed for user `b69b177927e14f80ae61c18791b25d27`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:53.354876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:53.418747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1295/2588] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:08:53.452362 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.454061 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:53.476723 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:53.517850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edb79f-c564-4df5-b83e-7d5fea3bf8fd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.521766 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f830094-71c6-413f-a848-d1e7edc0e4e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.522372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f830094-71c6-413f-a848-d1e7edc0e4e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.522654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f830094-71c6-413f-a848-d1e7edc0e4e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.522930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f830094-71c6-413f-a848-d1e7edc0e4e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.531700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f830094-71c6-413f-a848-d1e7edc0e4e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.533147 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f830094-71c6-413f-a848-d1e7edc0e4e3 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.541886 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1294/2589] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:08:53.557003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.577067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:53.619454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df25e5d-2e2f-48d3-b57a-72102a192a13 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.624504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.625129 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.625410 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.625683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] PATH_INFO: `/v3/users/b69b177927e14f80ae61c18791b25d27` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.626809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69b177927e14f80ae61c18791b25d27)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.628747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.676222 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69b177927e14f80ae61c18791b25d27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:53.683066 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69b177927e14f80ae61c18791b25d27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:53.694953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69b177927e14f80ae61c18791b25d27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:53.695615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69b177927e14f80ae61c18791b25d27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:53.702980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69b177927e14f80ae61c18791b25d27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:53.708781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1296/2590] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:53 2022] DELETE /identity/v3/users/b69b177927e14f80ae61c18791b25d27 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:53.713405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.713697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.713970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.733816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:53.734938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:53.794128 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1295/2591] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:53.804370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:53.805884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:53.865115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eb01c-a1b0-41d6-bc62-534f5eee1a4d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:53.868822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c28b7-61b7-4983-9716-0b0e9bea9025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:53.869435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c28b7-61b7-4983-9716-0b0e9bea9025 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:53.869718 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c28b7-61b7-4983-9716-0b0e9bea9025 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:53.869993 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c28b7-61b7-4983-9716-0b0e9bea9025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:53.880065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c28b7-61b7-4983-9716-0b0e9bea9025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:53.881521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c28b7-61b7-4983-9716-0b0e9bea9025 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:53.891454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1297/2592] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:54.166148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.167663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:54.185949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1dd3e-81d7-45e4-adff-05041ba4df4c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.191579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.192231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.192624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.192900 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] PATH_INFO: `/v3/projects/95bcf62929e24586a49f46020470c975` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.194038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95bcf62929e24586a49f46020470c975)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.195969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.258124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95bcf62929e24586a49f46020470c975', 'request_id': 'req-bf36054b-da8a-494b-9daa-3aae4b980e50'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:54.266809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95bcf62929e24586a49f46020470c975', 'request_id': 'req-bf36054b-da8a-494b-9daa-3aae4b980e50'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:54.274664 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95bcf62929e24586a49f46020470c975', 'request_id': 'req-bf36054b-da8a-494b-9daa-3aae4b980e50'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:54.287827 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] The token cache is being invalidate because project 95bcf62929e24586a49f46020470c975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:54.288017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:54.288678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bf36054b-da8a-494b-9daa-3aae4b980e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:54.290517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1296/2593] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:54 2022] DELETE /identity/v3/projects/95bcf62929e24586a49f46020470c975 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:54.296209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.296598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.296940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.318496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:54.319828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:54.357775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:54.385029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1298/2594] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:54.392937 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.393225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.393503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.413999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:54.415205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:54.423837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1297/2595] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:54.427943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.437778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fa1e9-ed12-4093-9272-874ecd467477 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.442330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.442921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.443202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.443474 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.444111 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:54.444455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.446042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.457252 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1299/2596] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:54.461424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.471195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21936ea9-9999-458f-9e38-cba7181fa772 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.475876 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.476507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.476799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.477073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.477699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.479586 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.507497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1298/2597] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:54 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:54.511475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.521865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc34d23-c579-42f6-aa43-1fe835506110 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.579295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.579940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.580224 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.580529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.581177 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.583154 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.605494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1300/2598] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:54 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:54.610223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.620474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c5e84-4b34-467b-a592-f1e384ad8f7c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.625139 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.625762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.626043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.626316 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.626953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.628577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.638433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1299/2599] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:08:54.642445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.652498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331383a-c253-4816-9414-78af58d19899 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.657870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaa115-f198-40f2-8af4-c980f071c0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.658599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa115-f198-40f2-8af4-c980f071c0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.658878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa115-f198-40f2-8af4-c980f071c0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.659149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa115-f198-40f2-8af4-c980f071c0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.659788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa115-f198-40f2-8af4-c980f071c0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.661405 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa115-f198-40f2-8af4-c980f071c0f9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.672704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1301/2600] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:54.677326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.687853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab037cb1-89bd-4180-ae3d-10cbdd4bc529 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.692638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.693296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.693890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.694177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] PATH_INFO: `/v3/projects/eb4602a0a13a4072a92f9a2623c1a8a3/users/bf35bbe2e81f4b028e83813fa7a9ede7/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.696615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:54.713192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4602a0a13a4072a92f9a2623c1a8a3, user_id=bf35bbe2e81f4b028e83813fa7a9ede7, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.716513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.730309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1300/2601] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:54 2022] PUT /identity/v3/projects/eb4602a0a13a4072a92f9a2623c1a8a3/users/bf35bbe2e81f4b028e83813fa7a9ede7/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:54.741276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.741587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.741852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.772597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] MFA Rules not processed for user `bf35bbe2e81f4b028e83813fa7a9ede7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:54.773861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:54.839397 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1302/2602] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:54.850711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:54.852556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:54.912960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f565e-bf0a-4d06-b66e-619264836181 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:54.916735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ad923-b55c-4700-a416-1002aa2d432b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:54.917366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ad923-b55c-4700-a416-1002aa2d432b service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:54.917630 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ad923-b55c-4700-a416-1002aa2d432b service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:54.917905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ad923-b55c-4700-a416-1002aa2d432b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:54.926502 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ad923-b55c-4700-a416-1002aa2d432b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:54.927927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ad923-b55c-4700-a416-1002aa2d432b service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:54.937422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1301/2603] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:57.994292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:58.035275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:58.062645 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137ab6-462a-4994-bd9a-e5863b30acd4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:58.067782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:58.068587 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.068970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.069401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] PATH_INFO: `/v3/users/bf35bbe2e81f4b028e83813fa7a9ede7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.070896 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf35bbe2e81f4b028e83813fa7a9ede7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:58.073199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:58.126270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf35bbe2e81f4b028e83813fa7a9ede7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.133127 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf35bbe2e81f4b028e83813fa7a9ede7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.145593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf35bbe2e81f4b028e83813fa7a9ede7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.146279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf35bbe2e81f4b028e83813fa7a9ede7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.153920 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf35bbe2e81f4b028e83813fa7a9ede7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.159822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1303/2604] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:08:57 2022] DELETE /identity/v3/users/bf35bbe2e81f4b028e83813fa7a9ede7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:58.164357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.164811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.164926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.184133 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:58.186058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:58.244556 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1302/2605] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:58.254125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:58.255522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:58.311849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396383-9014-4973-a562-1558b00ed06d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:58.315482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668baae-d72a-4769-8638-f19fcdd651e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:58.316078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668baae-d72a-4769-8638-f19fcdd651e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.316398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668baae-d72a-4769-8638-f19fcdd651e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.316665 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668baae-d72a-4769-8638-f19fcdd651e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.325579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668baae-d72a-4769-8638-f19fcdd651e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:58.327170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668baae-d72a-4769-8638-f19fcdd651e3 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:58.336575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1304/2606] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:08:58.620669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:58.622633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:58.635859 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9661c-a134-4a68-a8b7-f3a8c025eaf1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:58.642460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:58.643163 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.643547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.643977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] PATH_INFO: `/v3/projects/eb4602a0a13a4072a92f9a2623c1a8a3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.645335 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4602a0a13a4072a92f9a2623c1a8a3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:58.647987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:58.697814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4602a0a13a4072a92f9a2623c1a8a3', 'request_id': 'req-cda456b2-4327-4331-95d7-8b13909e4794'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.703698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4602a0a13a4072a92f9a2623c1a8a3', 'request_id': 'req-cda456b2-4327-4331-95d7-8b13909e4794'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.709867 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4602a0a13a4072a92f9a2623c1a8a3', 'request_id': 'req-cda456b2-4327-4331-95d7-8b13909e4794'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.722906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] The token cache is being invalidate because project eb4602a0a13a4072a92f9a2623c1a8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:08:58.723204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.723838 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cda456b2-4327-4331-95d7-8b13909e4794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:08:58.725353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1303/2607] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:08:58 2022] DELETE /identity/v3/projects/eb4602a0a13a4072a92f9a2623c1a8a3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:58.730259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.730548 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.730826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.751117 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:58.752246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:58.802954 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:58.814452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1305/2608] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:58.823287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.823662 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.823940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.846267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:58.847622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:58.855559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1304/2609] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:58.860543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:58.870115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a824-4bdf-4d6d-b097-7b0c949de6b0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:58.874962 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:58.875558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.875845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.876122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.876794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:08:58.877099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:58.878693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:58.889987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1306/2610] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:08:58.894076 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:58.899007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:08:58.906368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83809d3d-f8c0-4fa2-a821-909511204554 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:58.912614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:58.913227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:58.913532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:58.913771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:58.914439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:58.916380 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:59.070972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1305/2611] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:08:58 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:59.076072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:59.087736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34af60-e81c-43ab-93fc-307ba5312065 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:59.149334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:59.149980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:59.150261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:59.150535 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:59.151166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:59.153132 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:59.182030 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1307/2612] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:08:59 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:08:59.183175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:59.194007 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cc1c6-9ab0-4d34-b86f-4ee815d59663 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:59.198975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:59.199582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:59.199863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:59.200710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:59.201485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:59.203089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:59.213864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1306/2613] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:59.218558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:59.229151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d046478-8b16-4453-a630-cc366f596edb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:59.233825 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da3fff-b367-4c59-919d-502e243bee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:59.234436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3fff-b367-4c59-919d-502e243bee71 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:59.234711 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3fff-b367-4c59-919d-502e243bee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:59.235005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3fff-b367-4c59-919d-502e243bee71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:59.235641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da3fff-b367-4c59-919d-502e243bee71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:59.237251 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da3fff-b367-4c59-919d-502e243bee71 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:59.251464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1308/2614] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:08:59.252669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:59.263296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7e283-c3a2-44be-b985-930b55a03599 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:59.268389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:59.269043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:59.269354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:59.269627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] PATH_INFO: `/v3/projects/9472f5867b964202af54362d390c7509/users/0fa77b2d0ace4ab3af23ec0a10e23ea1/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:59.284642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9472f5867b964202af54362d390c7509, user_id=0fa77b2d0ace4ab3af23ec0a10e23ea1, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:59.287274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:59.300697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1307/2615] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:08:59 2022] PUT /identity/v3/projects/9472f5867b964202af54362d390c7509/users/0fa77b2d0ace4ab3af23ec0a10e23ea1/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:08:59.305706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcf514-48d6-4519-a2e3-b793a4b24f64 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:59.306091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcf514-48d6-4519-a2e3-b793a4b24f64 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:59.306431 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcf514-48d6-4519-a2e3-b793a4b24f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:59.326894 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a4dcf514-48d6-4519-a2e3-b793a4b24f64 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:08:59.328145 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a4dcf514-48d6-4519-a2e3-b793a4b24f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:59.387183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1309/2616] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:08:59.397880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:08:59.399587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:08:59.457889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b19d-18b9-4b07-bfa6-9fd77f010bd4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:08:59.461569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:08:59.462177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:08:59.462458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:08:59.462731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:08:59.472206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:08:59.473652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:08:59.483905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1308/2617] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:00.262379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcacc1-bb73-4638-82c8-b0c967c02621 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:00.263655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcacc1-bb73-4638-82c8-b0c967c02621 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:00.263655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcacc1-bb73-4638-82c8-b0c967c02621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:00.292402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-91fcacc1-bb73-4638-82c8-b0c967c02621 None None] MFA Rules not processed for user `0fa77b2d0ace4ab3af23ec0a10e23ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:00.293620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-91fcacc1-bb73-4638-82c8-b0c967c02621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:00.351680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:00.363452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1310/2618] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:00.375318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:00.376797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:00.435786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9d6ef-c379-4151-9068-612652bd3d6e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:00.439683 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:00.440611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:00.440973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:00.441273 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:00.449908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:00.451628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:00.460319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1309/2619] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:01.815156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:01.815668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:01.815668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:01.835197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] MFA Rules not processed for user `0fa77b2d0ace4ab3af23ec0a10e23ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:01.836418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:01.844941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1311/2620] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:08.189925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:08.198613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad263dc5-7df0-4d99-9a1f-100c83cb9734 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:08.202990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:08.203582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:08.203864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:08.204160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:08.205889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:08.256483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:08.286792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:08.288224 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:08.306724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1310/2621] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:09.205907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:09.207660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:09.223346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081883c2-0dae-47ab-babc-da1aaf2ad02f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:09.225597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:09.226788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:09.227076 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:09.227381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:09.229183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:09.232230 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:09.240257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:09.241706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:09.243701 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8419cb-e803-4249-853d-f7962ad4fe96 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:09.247647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:09.248296 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:09.248580 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:09.248854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:09.250599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:09.251249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1312/2622] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:09.264350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:09.267338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:09.277432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1311/2623] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:09:13.022037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:13.036866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70535a-200b-4d38-b267-f24c58ff7c76 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:13.044648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:13.045656 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:13.046756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:13.047248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:13.051684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:13.212503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:13.214441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:13.226794 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1313/2624] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:13.678679 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:13.680756 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:13.694112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4723506-8f25-4645-9978-e098876eeb75 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:13.699399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:13.701029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:13.701467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:13.702557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] PATH_INFO: `/v3/users/cd22d2df9ced4bb0961596385854422d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:13.717712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd22d2df9ced4bb0961596385854422d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:13.719662 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:13.775720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd22d2df9ced4bb0961596385854422d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:13.790764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd22d2df9ced4bb0961596385854422d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:13.806267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd22d2df9ced4bb0961596385854422d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:13.835022 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd22d2df9ced4bb0961596385854422d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:13.836942 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-654dfc74-88ee-4755-84e8-74ecf0bf5d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd22d2df9ced4bb0961596385854422d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:13.846259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1312/2625] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:13 2022] DELETE /identity/v3/users/cd22d2df9ced4bb0961596385854422d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:14.124067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.139865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac774a-d498-4510-9582-1c885f1cd42e service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.225659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.226815 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.227306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.227768 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] PATH_INFO: `/v3/projects/2e4c4dc894bd429b8f46ae444eb5ff7d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.241528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e4c4dc894bd429b8f46ae444eb5ff7d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.243593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.307999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e4c4dc894bd429b8f46ae444eb5ff7d', 'request_id': 'req-d2c3ffa4-4b54-43ad-8924-2149862abe27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:14.317425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e4c4dc894bd429b8f46ae444eb5ff7d', 'request_id': 'req-d2c3ffa4-4b54-43ad-8924-2149862abe27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:14.322240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:14.328358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e4c4dc894bd429b8f46ae444eb5ff7d', 'request_id': 'req-d2c3ffa4-4b54-43ad-8924-2149862abe27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:14.345366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] The token cache is being invalidate because project 2e4c4dc894bd429b8f46ae444eb5ff7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:14.345720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:14.346355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2c3ffa4-4b54-43ad-8924-2149862abe27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:14.351565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1314/2626] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:14 2022] DELETE /identity/v3/projects/2e4c4dc894bd429b8f46ae444eb5ff7d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:14.354165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.354432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.354707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.376815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:14.378292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:14.444759 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1313/2627] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:14.453638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799aced-b395-4d63-b412-b57253a7964e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.453928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799aced-b395-4d63-b412-b57253a7964e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.454206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799aced-b395-4d63-b412-b57253a7964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.474232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e799aced-b395-4d63-b412-b57253a7964e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:14.475473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e799aced-b395-4d63-b412-b57253a7964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:14.484180 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1315/2628] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:14.488475 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.499606 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe2ff6-1100-4304-911e-829c4f859784 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.507063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.507758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.508100 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.508490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.509413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:14.509787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.511468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.514987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:14.528466 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1314/2629] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:14.533023 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799aced-b395-4d63-b412-b57253a7964e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.542484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799aced-b395-4d63-b412-b57253a7964e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.547823 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.548470 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.548751 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.549713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.549713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.551553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.583379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1316/2630] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:14 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:14.587403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.598612 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389e55a-32d1-4898-9a35-22b24c3b2bdc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.685958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.686803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.687021 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.687631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.688322 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.690402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.720009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1315/2631] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:14 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:14.723620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.741146 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4e0bb-5a32-45cc-8e73-a923ef1382d5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.748733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.749424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.749709 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.749975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.750618 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.753676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.772555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1317/2632] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:14.774162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.785372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36808263-8350-472d-ab26-4b9d6d08e360 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.790071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b951ce-f9a3-4b17-a854-92e752ac33ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.790685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b951ce-f9a3-4b17-a854-92e752ac33ee admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.790973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b951ce-f9a3-4b17-a854-92e752ac33ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.791306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b951ce-f9a3-4b17-a854-92e752ac33ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.791948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b951ce-f9a3-4b17-a854-92e752ac33ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.793745 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b951ce-f9a3-4b17-a854-92e752ac33ee admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.804948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1316/2633] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:14.810715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:14.821843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6395d7-516d-4b67-9587-feb87a32daad admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:14.829138 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:14.829961 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.830256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.831299 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] PATH_INFO: `/v3/projects/f9099ca971d84b27be7f46af20a52352/users/e308a6ca7776433780443a1f6a6cc890/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.859882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9099ca971d84b27be7f46af20a52352, user_id=e308a6ca7776433780443a1f6a6cc890, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:14.863381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:14.888314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1318/2634] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:14 2022] PUT /identity/v3/projects/f9099ca971d84b27be7f46af20a52352/users/e308a6ca7776433780443a1f6a6cc890/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:14.897302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:14.898091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:14.898379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:14.929618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] MFA Rules not processed for user `e308a6ca7776433780443a1f6a6cc890`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:14.930834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:14.996488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1317/2635] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:15.001361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.113392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b578bf-ca26-4b41-a877-9b650267c888 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.119623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.121046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.121389 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.122247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.122983 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.125213 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.181336 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1319/2636] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:15 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:15.186170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.203517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b880-3fa2-4161-942e-5d1afabdb210 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.282735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.283386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.283801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.284082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.284751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.286654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.297710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:15.315410 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1318/2637] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:15 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:15.319579 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.332980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89c344-1bdc-43db-a320-427a58467f17 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.338804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.339427 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.339710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.339987 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.341228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.342856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.362147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1320/2638] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:15.365642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.377241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fda2ce-ab80-4931-a1eb-f0795b268c42 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.381842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.382506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.382793 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.383070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] PATH_INFO: `/v3/projects/399453798ebb4da4add1f838b4f122c3/users/fe0871076bf04b26a1217f0554baeb60/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.402528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399453798ebb4da4add1f838b4f122c3, user_id=fe0871076bf04b26a1217f0554baeb60, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.406702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.422554 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1319/2639] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:15 2022] PUT /identity/v3/projects/399453798ebb4da4add1f838b4f122c3/users/fe0871076bf04b26a1217f0554baeb60/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:15.426783 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.437551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d89b6e-60eb-44ce-8cf0-8a363bf4bafd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.495892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4b53b-f57d-4fc1-800a-4329665c160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.496558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4b53b-f57d-4fc1-800a-4329665c160e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.496938 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4b53b-f57d-4fc1-800a-4329665c160e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.497260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4b53b-f57d-4fc1-800a-4329665c160e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.498099 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4b53b-f57d-4fc1-800a-4329665c160e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.499875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4b53b-f57d-4fc1-800a-4329665c160e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.501446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:15.516788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1321/2640] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:15.518658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.532248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2ba6c-49f4-44d4-aac2-affe16511258 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.538084 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.538851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.539135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.539406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] PATH_INFO: `/v3/projects/399453798ebb4da4add1f838b4f122c3/users/fe0871076bf04b26a1217f0554baeb60/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.541536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] RBAC: Authorizing `identity:create_grant(project_id=399453798ebb4da4add1f838b4f122c3, user_id=fe0871076bf04b26a1217f0554baeb60, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.545115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.560365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1320/2641] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:15 2022] PUT /identity/v3/projects/399453798ebb4da4add1f838b4f122c3/users/fe0871076bf04b26a1217f0554baeb60/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:15.569149 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.569438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.569778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.601795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] MFA Rules not processed for user `fe0871076bf04b26a1217f0554baeb60`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:15.603018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:15.685965 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1322/2642] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:15.697615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.701247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:15.821564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c64050-2e18-4c02-9d3f-6a901445252b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.826950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.827651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.827999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.828367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:15.845359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:15.847002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:15.860202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1321/2643] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:15.924743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:15.926594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:15.940508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525d686-d477-4415-b1e9-d327b67a8d0d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:15.944850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:15.945552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:15.946312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:15.946312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:16.016913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:16.018731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:16.029611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1323/2644] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:18.238857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:18.240320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:18.350361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee206-fbe9-4f25-9364-d3541d1f03e3 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:18.357505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:18.364643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:18.365042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:18.365425 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:18.372962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:18.400134 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:18.404023 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea61abd-0871-48cb-906b-d63e0a1cdfc1 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:18.429698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1322/2645] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:19.605219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:19.677431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:19.677988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:19.678091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:19.686780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b5dba-e7b6-4247-b7ec-3ad7842e8588 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:19.692786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:19.694116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:19.694479 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:19.694991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] PATH_INFO: `/v3/users/e308a6ca7776433780443a1f6a6cc890` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:19.696595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e308a6ca7776433780443a1f6a6cc890)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:19.702046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:19.706004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] MFA Rules not processed for user `0fa77b2d0ace4ab3af23ec0a10e23ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:19.706004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:19.727044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:19.755956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:19.761298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e308a6ca7776433780443a1f6a6cc890'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:19.768763 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e308a6ca7776433780443a1f6a6cc890'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:19.773340 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1323/2646] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:09:19.774311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:19.775705 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:19.786434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e308a6ca7776433780443a1f6a6cc890'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:19.787232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e308a6ca7776433780443a1f6a6cc890'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:19.795897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3dcfe59-1464-44fb-9136-517f18617399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e308a6ca7776433780443a1f6a6cc890'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:19.803025 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1324/2647] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:19 2022] DELETE /identity/v3/users/e308a6ca7776433780443a1f6a6cc890 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:19.807284 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:19.807674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:19.808044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:19.830530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:19.832528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:19.842218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddce7ad-8c33-48a6-8a61-b7cada489ba3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:19.896050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:19.896964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:19.897247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:19.897514 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:19.899231 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:19.904211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1325/2648] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:09:19.910985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:19.913087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:19.916635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:19.918194 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:19.929101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1324/2649] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:19.929940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e37608-02fe-412b-acd1-526c109ba896 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:19.934467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:19.935080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:19.935354 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:19.935625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:19.944948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:19.946384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:19.955960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1326/2650] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:09:20.763801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:20.765600 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:20.778512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a66a3-8f11-4d67-a032-b80f2f0730ac service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:20.783328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:20.784034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:20.784436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:20.784962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] PATH_INFO: `/v3/projects/f9099ca971d84b27be7f46af20a52352` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:20.787059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9099ca971d84b27be7f46af20a52352)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:20.789765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:20.846215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9099ca971d84b27be7f46af20a52352', 'request_id': 'req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:20.852483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9099ca971d84b27be7f46af20a52352', 'request_id': 'req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:20.859842 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9099ca971d84b27be7f46af20a52352', 'request_id': 'req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:20.874746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] The token cache is being invalidate because project f9099ca971d84b27be7f46af20a52352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:20.875182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:20.876028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:20.880949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1325/2651] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:20 2022] DELETE /identity/v3/projects/f9099ca971d84b27be7f46af20a52352 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:20.882019 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:20.892399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33819489-021f-466f-b6dc-aa0ab36d269c service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:20.973369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:20.974016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:20.974507 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:20.974935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] PATH_INFO: `/v3/users/fe0871076bf04b26a1217f0554baeb60` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:20.976244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0871076bf04b26a1217f0554baeb60)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:20.978357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:21.029286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:21.059441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0871076bf04b26a1217f0554baeb60'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.068314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0871076bf04b26a1217f0554baeb60'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.088197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0871076bf04b26a1217f0554baeb60'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.089268 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0871076bf04b26a1217f0554baeb60'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.105225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b56f7073-56a2-4873-b913-2bcaa9b2e3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0871076bf04b26a1217f0554baeb60'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.114666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1327/2652] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:20 2022] DELETE /identity/v3/users/fe0871076bf04b26a1217f0554baeb60 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:21.456930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:21.458416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:21.473051 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6b5c-70b5-4c96-a77e-01125d8c125e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:21.531267 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:21.565074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:21.565729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:21.566073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:21.566411 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] PATH_INFO: `/v3/projects/399453798ebb4da4add1f838b4f122c3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:21.567948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399453798ebb4da4add1f838b4f122c3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:21.570027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:21.638248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399453798ebb4da4add1f838b4f122c3', 'request_id': 'req-78864b9d-7510-4027-adef-4275c63719e8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.646014 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399453798ebb4da4add1f838b4f122c3', 'request_id': 'req-78864b9d-7510-4027-adef-4275c63719e8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.654196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399453798ebb4da4add1f838b4f122c3', 'request_id': 'req-78864b9d-7510-4027-adef-4275c63719e8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.672071 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] The token cache is being invalidate because project 399453798ebb4da4add1f838b4f122c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:21.673610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.676392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-78864b9d-7510-4027-adef-4275c63719e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:21.682623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1326/2653] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:21 2022] DELETE /identity/v3/projects/399453798ebb4da4add1f838b4f122c3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:21.685351 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:21.685898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:21.686641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:21.708403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:21.709646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:21.778003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1328/2654] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:21.786404 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:21.786927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:21.786927 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:21.808342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:21.810215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:21.819368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1327/2655] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:21.823497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:21.834550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfe2f1-4a11-4382-9079-70da6098d079 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:21.839804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:21.840766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:21.841100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:21.841402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:21.842056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:21.842349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:21.844069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:21.860833 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1329/2656] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:21.865197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:21.874624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a32689-6f08-4545-ac3f-1d1de20e2cf9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:21.879292 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:21.879896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:21.880172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:21.880507 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:21.881146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:21.883018 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:21.912565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1328/2657] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:21 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:21.917037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:21.927291 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfbb51f-c050-4147-a03d-aa77abef5e91 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:21.985157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:21.985781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:21.986067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:21.986342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:21.987592 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:21.989577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.017142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1330/2658] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:21 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:22.021408 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:22.035555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb9c2b-22e5-485d-a99e-88461468feac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:22.042063 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:22.042780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.042952 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.045849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.046709 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:22.049464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.078080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:22.105540 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1329/2659] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:22.109838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d8f4b-d30e-4dc6-8269-0b3dfae12d1b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:22.115029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82791af-d3a5-43c4-bbe4-53c88f072da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:22.115660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82791af-d3a5-43c4-bbe4-53c88f072da5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.115945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82791af-d3a5-43c4-bbe4-53c88f072da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.116225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82791af-d3a5-43c4-bbe4-53c88f072da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.116936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82791af-d3a5-43c4-bbe4-53c88f072da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:22.118926 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82791af-d3a5-43c4-bbe4-53c88f072da5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.138490 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:22.144499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1331/2660] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:22.158351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3f7c5-60a8-4a91-aef8-0143accbae44 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:22.168977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:22.169494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.170004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.170486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] PATH_INFO: `/v3/projects/d23ed795748945a787c6ee5dec7a5e8f/users/9da9232626ae473ba0c78a9ea4a8491e/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.187459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23ed795748945a787c6ee5dec7a5e8f, user_id=9da9232626ae473ba0c78a9ea4a8491e, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:22.191610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.208343 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1330/2661] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:22 2022] PUT /identity/v3/projects/d23ed795748945a787c6ee5dec7a5e8f/users/9da9232626ae473ba0c78a9ea4a8491e/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:22.219738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.220260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.220555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.244820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:22.264528 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] MFA Rules not processed for user `9da9232626ae473ba0c78a9ea4a8491e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:22.266085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:22.372776 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1332/2662] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:22.382611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:22.386567 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:22.468591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c6f03-0dbd-4cbe-80c9-cd0ed939c027 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:22.492752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9b293-ef12-4ca0-95bf-146b0de95a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:22.493704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9b293-ef12-4ca0-95bf-146b0de95a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.493995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9b293-ef12-4ca0-95bf-146b0de95a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.494268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9b293-ef12-4ca0-95bf-146b0de95a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.505020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9b293-ef12-4ca0-95bf-146b0de95a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:22.506828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9b293-ef12-4ca0-95bf-146b0de95a0c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.512799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:22.520823 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1331/2663] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:22.548150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:22.627553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bf208-61af-46d0-93e5-56b917bd9139 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:22.632865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:22.633042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.633296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.634188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] PATH_INFO: `/v3/users/9da9232626ae473ba0c78a9ea4a8491e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.634973 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9da9232626ae473ba0c78a9ea4a8491e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:22.638213 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.707620 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9da9232626ae473ba0c78a9ea4a8491e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:22.716601 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9da9232626ae473ba0c78a9ea4a8491e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:22.732419 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9da9232626ae473ba0c78a9ea4a8491e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:22.733216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9da9232626ae473ba0c78a9ea4a8491e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:22.743216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9da9232626ae473ba0c78a9ea4a8491e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:22.766848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1333/2664] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:22 2022] DELETE /identity/v3/users/9da9232626ae473ba0c78a9ea4a8491e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:22.768186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.768505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.768862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.801117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:22.803772 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:22.869973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1332/2665] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:22.882826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:22.884651 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:22.910812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:22.959500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b210f-5e72-4fa1-8378-4783241dfdef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:22.963654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:22.964314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:22.964604 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:22.964899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:22.974631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:22.976732 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:22.988969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1334/2666] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:24.056398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.059613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.089449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82034131-0f7b-4a8f-b8d1-38539fa74c54 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.096719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.097685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.098739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.099291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] PATH_INFO: `/v3/users/2f0feb7134ee408b989b398a315850fe` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.101132 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0feb7134ee408b989b398a315850fe)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.103038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.104878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.108089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.125680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753548-8c6a-4c89-9695-943dba11b198 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.139710 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.140876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.141376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.142803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] PATH_INFO: `/v3/projects/d23ed795748945a787c6ee5dec7a5e8f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.145770 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23ed795748945a787c6ee5dec7a5e8f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.149331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.177118 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0feb7134ee408b989b398a315850fe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.193925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0feb7134ee408b989b398a315850fe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.205274 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0feb7134ee408b989b398a315850fe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.208492 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23ed795748945a787c6ee5dec7a5e8f', 'request_id': 'req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.214181 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0feb7134ee408b989b398a315850fe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.215620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e6e5eaa5-ebff-4c77-a932-f997992902e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0feb7134ee408b989b398a315850fe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.215810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23ed795748945a787c6ee5dec7a5e8f', 'request_id': 'req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.222636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1333/2667] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:24 2022] DELETE /identity/v3/users/2f0feb7134ee408b989b398a315850fe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:24.222795 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23ed795748945a787c6ee5dec7a5e8f', 'request_id': 'req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.239344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] The token cache is being invalidate because project d23ed795748945a787c6ee5dec7a5e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:24.239657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.240615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d9f9a9bf-5997-4cc2-9de4-a982d9a07082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.242135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1335/2668] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:24 2022] DELETE /identity/v3/projects/d23ed795748945a787c6ee5dec7a5e8f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:24.247644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.248009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.248522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.270898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:24.272689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.341504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1334/2669] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:24.350446 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.350950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.351539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.373323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:24.374583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.383510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1336/2670] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:24.388306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.398734 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ea8b-8f3b-4ac2-be62-babe39040a61 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.404729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.405503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.405834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.406113 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.406804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:24.407109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.408792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.418950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:24.426382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1335/2671] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:24.427533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.434846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.437270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.438352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039ecd0-5484-4b06-8d1e-e0b067b3a184 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.452013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c003-19d4-455d-b7e6-881b9af31ef7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.452518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.454018 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.456184 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.456553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.457316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.458477 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.459125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.459413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.459852 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] PATH_INFO: `/v3/projects/6180da28285943f7a7b0900185df6127` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.460596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.461251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6180da28285943f7a7b0900185df6127)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.463345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.508142 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1337/2672] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:24.512401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.548935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6180da28285943f7a7b0900185df6127', 'request_id': 'req-85c8d7f3-9fee-4103-916f-4240cb168469'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.555402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6180da28285943f7a7b0900185df6127', 'request_id': 'req-85c8d7f3-9fee-4103-916f-4240cb168469'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.566102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6180da28285943f7a7b0900185df6127', 'request_id': 'req-85c8d7f3-9fee-4103-916f-4240cb168469'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.579374 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] The token cache is being invalidate because project 6180da28285943f7a7b0900185df6127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:24.579674 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.580367 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-85c8d7f3-9fee-4103-916f-4240cb168469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:24.580968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0ce05-7126-4146-b787-2e91820e7934 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.582038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1336/2673] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:24 2022] DELETE /identity/v3/projects/6180da28285943f7a7b0900185df6127 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:24.586713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb080770-df58-42bc-8443-333da5337244 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.586997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb080770-df58-42bc-8443-333da5337244 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.587269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb080770-df58-42bc-8443-333da5337244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.604339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:24.614395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eb080770-df58-42bc-8443-333da5337244 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:24.615711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eb080770-df58-42bc-8443-333da5337244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.661042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a8e7c-ae6c-4f5c-8c8e-fb69d46b90cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.661841 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a8e7c-ae6c-4f5c-8c8e-fb69d46b90cc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.662217 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a8e7c-ae6c-4f5c-8c8e-fb69d46b90cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.662835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a8e7c-ae6c-4f5c-8c8e-fb69d46b90cc admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.663627 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a8e7c-ae6c-4f5c-8c8e-fb69d46b90cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.665979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a8e7c-ae6c-4f5c-8c8e-fb69d46b90cc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.692372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1337/2674] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:24.692596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1338/2675] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:24.697534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb080770-df58-42bc-8443-333da5337244 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.697912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.698196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.698586 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.699264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eb080770-df58-42bc-8443-333da5337244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.712689 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb080770-df58-42bc-8443-333da5337244 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.717926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.718538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.718813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.719080 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.719853 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.721541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:24.721693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.724034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.733360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1339/2676] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:09:24.736873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1338/2677] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:24.738232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.738684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.749141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b18d7-f9dc-436e-9fb4-09086afe60ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.753921 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63853cb1-d210-4d4a-9cbc-42015fd05768 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.756420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.757226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.757441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.757707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.758369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:24.758673 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.760258 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.760500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.761158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.761448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.761720 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.763199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.765504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.783343 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1340/2678] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:09:24.783555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1339/2679] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:24.783693 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.784387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.795415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fddf2-5bdf-4b2a-bddc-0cc306a1bedc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.795995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388beec3-ce58-4e0b-a086-b648f276f17e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.800318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.800959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.801237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.801511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.801741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d47ee-09d4-4179-814c-0265c60c5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.802131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.802738 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d47ee-09d4-4179-814c-0265c60c5126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.803027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d47ee-09d4-4179-814c-0265c60c5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.803312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d47ee-09d4-4179-814c-0265c60c5126 admin admin] PATH_INFO: `/v3/projects/f3a4a0ea38d64717b1579c1362aa3d0a/users/d166424a556e47e3b8ddee852d10b657/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.804098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.820263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d47ee-09d4-4179-814c-0265c60c5126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a4a0ea38d64717b1579c1362aa3d0a, user_id=d166424a556e47e3b8ddee852d10b657, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.823398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1d47ee-09d4-4179-814c-0265c60c5126 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.838240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1340/2680] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:24.841140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1341/2681] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:24 2022] PUT /identity/v3/projects/f3a4a0ea38d64717b1579c1362aa3d0a/users/d166424a556e47e3b8ddee852d10b657/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:24.843087 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.852196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.852636 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.852927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.883643 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] MFA Rules not processed for user `d166424a556e47e3b8ddee852d10b657`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:24.884986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.897993 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:24.934538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996f0af-a6b8-4d63-9bf5-a133eac8606c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.939941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.940332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.940621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.940910 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.941523 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.943795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:24.959089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1342/2682] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:09:24.972086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1341/2683] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:24 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:24.972750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.974269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:24.975760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:24.986754 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59894b-a7c6-4d1a-b6ee-e26537a0be89 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:24.992294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:24.992914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:24.993199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:24.993467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:24.994365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:24.995802 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:25.006908 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1342/2684] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:25.010837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:25.022300 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609d5b-d033-4276-89d6-b18d5a1f463d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:25.028640 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:25.029253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:25.029538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:25.029809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:25.030965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:25.032032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:25.042110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce0f72-01ba-4e54-9351-d4be1280d03e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:25.043646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1343/2685] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:25.046913 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf2592-0ef7-4722-b4e8-55d2bdd15241 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e29455534ee474899b4f470d2a55f94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:25.047530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2592-0ef7-4722-b4e8-55d2bdd15241 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:25.047810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2592-0ef7-4722-b4e8-55d2bdd15241 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:25.048077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2592-0ef7-4722-b4e8-55d2bdd15241 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:25.048195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:25.056810 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf2592-0ef7-4722-b4e8-55d2bdd15241 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:25.058231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf2592-0ef7-4722-b4e8-55d2bdd15241 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:25.058356 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa3d26-76ca-428b-854d-c8a07e5b7788 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:25.063749 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:25.064442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:25.064712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:25.064989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] PATH_INFO: `/v3/projects/bc559da8928a4f909f86dba1c455472c/users/2687e887c3f349fbae1bd1d83b4e7a67/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:25.068907 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1343/2686] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:09:25.083269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc559da8928a4f909f86dba1c455472c, user_id=2687e887c3f349fbae1bd1d83b4e7a67, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:25.086028 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:25.100973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1344/2687] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:25 2022] PUT /identity/v3/projects/bc559da8928a4f909f86dba1c455472c/users/2687e887c3f349fbae1bd1d83b4e7a67/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:25.110928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:25.111313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:25.111483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:25.118772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:25.145494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] MFA Rules not processed for user `2687e887c3f349fbae1bd1d83b4e7a67`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:25.147244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:25.219096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1344/2688] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:25.230043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:25.231971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:25.308101 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf17add-17e8-40af-889b-b91d79da8de8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:25.314795 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c31461-7ee8-4135-a6ab-9296523bb332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:25.315405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c31461-7ee8-4135-a6ab-9296523bb332 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:25.315697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c31461-7ee8-4135-a6ab-9296523bb332 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:25.315969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c31461-7ee8-4135-a6ab-9296523bb332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:25.323984 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c31461-7ee8-4135-a6ab-9296523bb332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:25.326227 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c31461-7ee8-4135-a6ab-9296523bb332 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:25.335336 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1345/2689] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:09:35.216015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=31 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:35.331276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=65,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:35.797208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:35.858871 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca0f44-0f02-4368-bed7-67a0b70c15ed None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:35.863359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:35.863985 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:35.864282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:35.864571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] PATH_INFO: `/v3/users/2687e887c3f349fbae1bd1d83b4e7a67` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:35.865716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2687e887c3f349fbae1bd1d83b4e7a67)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:35.867640 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:35.914199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2687e887c3f349fbae1bd1d83b4e7a67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:35.921875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2687e887c3f349fbae1bd1d83b4e7a67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:35.934313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2687e887c3f349fbae1bd1d83b4e7a67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:35.935054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2687e887c3f349fbae1bd1d83b4e7a67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:35.942731 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2687e887c3f349fbae1bd1d83b4e7a67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:35.948724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1345/2690] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:35 2022] DELETE /identity/v3/users/2687e887c3f349fbae1bd1d83b4e7a67 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:35.953247 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:35.953544 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:35.953826 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:35.974236 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:35.975467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:36.044193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1346/2691] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:36.052110 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.053820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:36.123865 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774901-e961-419e-ba12-8f107aec92ce admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.127686 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa50b0-bcab-47e9-a1d7-6cce860355eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.128328 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa50b0-bcab-47e9-a1d7-6cce860355eb service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.128617 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa50b0-bcab-47e9-a1d7-6cce860355eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.128895 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa50b0-bcab-47e9-a1d7-6cce860355eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.138034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa50b0-bcab-47e9-a1d7-6cce860355eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.140204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa50b0-bcab-47e9-a1d7-6cce860355eb service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.151881 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1346/2692] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:36.424922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.426431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:36.440949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcda1c4-6738-4155-b629-eeb4678daac0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.445534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.446174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.446460 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.446775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] PATH_INFO: `/v3/projects/bc559da8928a4f909f86dba1c455472c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.447888 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc559da8928a4f909f86dba1c455472c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.449905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.500222 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc559da8928a4f909f86dba1c455472c', 'request_id': 'req-c6bd8cc5-2065-41b7-a55a-2f443e32768f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:36.506822 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc559da8928a4f909f86dba1c455472c', 'request_id': 'req-c6bd8cc5-2065-41b7-a55a-2f443e32768f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:36.513456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc559da8928a4f909f86dba1c455472c', 'request_id': 'req-c6bd8cc5-2065-41b7-a55a-2f443e32768f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:36.534783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] The token cache is being invalidate because project bc559da8928a4f909f86dba1c455472c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:36.535077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:36.536700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c6bd8cc5-2065-41b7-a55a-2f443e32768f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:36.542461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1347/2693] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:36 2022] DELETE /identity/v3/projects/bc559da8928a4f909f86dba1c455472c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:36.544231 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.544715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.544818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.566369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:36.567583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:36.587057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:36.634330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1347/2694] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:36.642918 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.643291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.643659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.665383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:36.666646 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:36.677189 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1348/2695] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:36.681737 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.692048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456d8f-3732-480f-b8a4-a503e2234c0e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.696835 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.697469 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.698087 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.698367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.699048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:36.699386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.701078 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.713296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1348/2696] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:36.717926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.729241 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3457d-8323-49f8-aa54-92c33b637c63 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.734740 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.735347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.735686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.735964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.736633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.738524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.766471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1349/2697] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:36 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:36.771042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.784558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da99f5-97b5-48a0-a9a0-30d34d9ea39e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.841544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.842809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.843193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.843367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.843995 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.845951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.866248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1349/2698] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:36 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:36.870431 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.883434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb5465-3e90-45a4-9406-917d6c574f57 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.888098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.888973 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.889383 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.889741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.890381 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.891959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.902831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1350/2699] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:36.907916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.917786 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8627493-8eef-4cdf-95ff-4d56a5031c5c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.922968 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7426c-3019-4dfb-a8e6-d9b51b4016d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.923557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7426c-3019-4dfb-a8e6-d9b51b4016d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.923838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7426c-3019-4dfb-a8e6-d9b51b4016d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.924107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7426c-3019-4dfb-a8e6-d9b51b4016d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.924772 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7426c-3019-4dfb-a8e6-d9b51b4016d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.926355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7426c-3019-4dfb-a8e6-d9b51b4016d5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.937484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1350/2700] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:36.941988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:36.953784 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea996bf5-2209-41e5-8e74-e06319c2298b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:36.958448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:36.959079 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:36.959358 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:36.959694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] PATH_INFO: `/v3/projects/760800fe875445c1a21050ee7e1805f9/users/b13b6dabfad045c9bc2e588479c87831/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:36.977043 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=760800fe875445c1a21050ee7e1805f9, user_id=b13b6dabfad045c9bc2e588479c87831, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:36.980257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:36.998614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1351/2701] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:36 2022] PUT /identity/v3/projects/760800fe875445c1a21050ee7e1805f9/users/b13b6dabfad045c9bc2e588479c87831/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:37.008991 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.009282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.009554 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.039370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] MFA Rules not processed for user `b13b6dabfad045c9bc2e588479c87831`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:37.040674 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:37.106901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1351/2702] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:37.111948 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.138124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:37.183880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9a8c8-e7f4-46a3-8edf-e23355f58a76 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.189291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.189843 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.190154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.190459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.191095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.193078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.220818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1352/2703] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:37 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:37.225339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.237244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ffd6c-9e20-43ac-b444-611a079fbc72 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.250376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:37.298624 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.299243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.299529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.299811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.300483 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.302400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.325061 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1352/2704] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:37 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:37.329729 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.340563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b74da-d837-4df3-aef4-12408112878c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.345877 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.346479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.346762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.347036 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.347673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.349295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.360198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1353/2705] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:37.364568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.374782 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d434e2-752d-4af0-9e77-32dd5f7d9be8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.380107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.381014 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.381298 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.381575 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] PATH_INFO: `/v3/projects/55d15474129e462fabb987fd9fb90f11/users/2680eb5fed0b41f681ad8fd977294167/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.397868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d15474129e462fabb987fd9fb90f11, user_id=2680eb5fed0b41f681ad8fd977294167, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.400570 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.413532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1353/2706] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:37 2022] PUT /identity/v3/projects/55d15474129e462fabb987fd9fb90f11/users/2680eb5fed0b41f681ad8fd977294167/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:37.417512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.429056 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f228d6d5-01b8-4e42-b9fb-848ae2ccb59f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.492661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047eb74a-d9c2-4674-bbce-b4f81782f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.493281 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eb74a-d9c2-4674-bbce-b4f81782f848 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.493564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eb74a-d9c2-4674-bbce-b4f81782f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.493971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047eb74a-d9c2-4674-bbce-b4f81782f848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.494622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047eb74a-d9c2-4674-bbce-b4f81782f848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.496245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047eb74a-d9c2-4674-bbce-b4f81782f848 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.506499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1354/2707] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:37.510634 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.521931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf9913-04b5-410c-b557-8c909e7434ee admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.526637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.527290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.527573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.527850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] PATH_INFO: `/v3/projects/55d15474129e462fabb987fd9fb90f11/users/2680eb5fed0b41f681ad8fd977294167/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.530322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d15474129e462fabb987fd9fb90f11, user_id=2680eb5fed0b41f681ad8fd977294167, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.533288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.544796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1354/2708] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:37 2022] PUT /identity/v3/projects/55d15474129e462fabb987fd9fb90f11/users/2680eb5fed0b41f681ad8fd977294167/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:37.557160 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.557458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.557739 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.587436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] MFA Rules not processed for user `2680eb5fed0b41f681ad8fd977294167`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:37.588700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:37.658638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1355/2709] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:37.670369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.672179 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:37.730004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397b16-2239-410d-b7bb-b8dd53b48b31 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.734303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1912f-c591-404c-8a6f-70b13589b97f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.734915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1912f-c591-404c-8a6f-70b13589b97f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.735204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1912f-c591-404c-8a6f-70b13589b97f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.735482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1912f-c591-404c-8a6f-70b13589b97f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.744939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1912f-c591-404c-8a6f-70b13589b97f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.746326 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1912f-c591-404c-8a6f-70b13589b97f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.756689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1355/2710] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:37.826527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:37.846899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:37.897161 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e21111-8ecc-4c79-ac70-aa628ec7cc6b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:37.901758 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:37.902485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.902906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.903240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] PATH_INFO: `/v3/users/b13b6dabfad045c9bc2e588479c87831` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:37.904956 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13b6dabfad045c9bc2e588479c87831)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:37.906872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:37.955419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13b6dabfad045c9bc2e588479c87831'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:37.969366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13b6dabfad045c9bc2e588479c87831'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:37.976726 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13b6dabfad045c9bc2e588479c87831'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:37.985435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13b6dabfad045c9bc2e588479c87831'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:37.986148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13b6dabfad045c9bc2e588479c87831'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:37.992038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1356/2711] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:37 2022] DELETE /identity/v3/users/b13b6dabfad045c9bc2e588479c87831 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:37.997045 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:37.997272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:37.997550 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:38.017316 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:38.018530 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:38.083371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1356/2712] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:38.090032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:38.091401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:38.154527 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f60bf3-c688-48bf-b513-1a7226a12902 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:38.159562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:38.160175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:38.160526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:38.160807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:38.170899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:38.172498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:38.186406 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1357/2713] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:38.470323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:38.471748 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:38.484910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35377028-405c-4d80-ac51-b624439c4a9c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:38.489588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:38.490219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:38.490495 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:38.490766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] PATH_INFO: `/v3/projects/760800fe875445c1a21050ee7e1805f9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:38.491892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=760800fe875445c1a21050ee7e1805f9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:38.493917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:38.503633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:38.549285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760800fe875445c1a21050ee7e1805f9', 'request_id': 'req-95a4244f-029d-4d08-b146-d3521549e2b0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.554952 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760800fe875445c1a21050ee7e1805f9', 'request_id': 'req-95a4244f-029d-4d08-b146-d3521549e2b0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.561323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760800fe875445c1a21050ee7e1805f9', 'request_id': 'req-95a4244f-029d-4d08-b146-d3521549e2b0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.574691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] The token cache is being invalidate because project 760800fe875445c1a21050ee7e1805f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:38.574691 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.575100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.577211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1357/2714] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:38 2022] DELETE /identity/v3/projects/760800fe875445c1a21050ee7e1805f9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:38.580989 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:38.591162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702de0d-d721-4f75-baf2-26053942280e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:38.625959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:38.652605 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:38.653246 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:38.653553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:38.653792 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] PATH_INFO: `/v3/users/2680eb5fed0b41f681ad8fd977294167` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:38.655049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2680eb5fed0b41f681ad8fd977294167)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:38.656992 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:38.712170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2680eb5fed0b41f681ad8fd977294167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.724926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2680eb5fed0b41f681ad8fd977294167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.732541 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2680eb5fed0b41f681ad8fd977294167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.741439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2680eb5fed0b41f681ad8fd977294167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.742064 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b7e945f2-b399-4260-84ba-909465f677cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2680eb5fed0b41f681ad8fd977294167'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:38.749115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1358/2715] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:38 2022] DELETE /identity/v3/users/2680eb5fed0b41f681ad8fd977294167 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:39.097616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.099611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:39.113917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4244f-029d-4d08-b146-d3521549e2b0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.173706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.174349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.174642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.174921 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] PATH_INFO: `/v3/projects/55d15474129e462fabb987fd9fb90f11` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.176046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d15474129e462fabb987fd9fb90f11)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.177994 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.224538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d15474129e462fabb987fd9fb90f11', 'request_id': 'req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:39.230230 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d15474129e462fabb987fd9fb90f11', 'request_id': 'req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:39.237171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d15474129e462fabb987fd9fb90f11', 'request_id': 'req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:39.250669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] The token cache is being invalidate because project 55d15474129e462fabb987fd9fb90f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:39.250960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:39.251588 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-afecf3cb-84c2-4393-80f0-aa814d7cdbdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:39.253125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1358/2716] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:39 2022] DELETE /identity/v3/projects/55d15474129e462fabb987fd9fb90f11 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:39.258192 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.258548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.258862 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.283904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:39.285196 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:39.344449 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1359/2717] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:39.352280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.352611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.352888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.371974 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:39.373778 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:39.381959 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1359/2718] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:39.386263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.396987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f8bd8-a729-456d-8dcf-80b7cae5eeb7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.401558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.403004 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.403293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.403568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.404252 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:39.404596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.406190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.417771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1360/2719] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:39.422275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.431348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba72d8-7836-4e3b-b0a5-9356da7207c2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.436421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.437006 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.437279 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.437544 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.438170 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.440042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.461445 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:39.471215 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1360/2720] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:39.475631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.485596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d98b7-e58f-48b1-aa51-ac9782213bdd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.526456 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:39.542786 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.543389 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.543669 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.543941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.544599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.546546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.567697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1361/2721] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:39.571868 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.582745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e839-e923-4922-ba4e-c57023965f29 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.587249 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.587840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.588113 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.588416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.589040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.590616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.600632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1361/2722] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:39.604593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.614504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24c347-86bf-489a-aafa-f48e93500532 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.619089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb5fa3-12e2-4a9f-bfc3-87059b88ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.619677 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb5fa3-12e2-4a9f-bfc3-87059b88ab8e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.619959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb5fa3-12e2-4a9f-bfc3-87059b88ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.620233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb5fa3-12e2-4a9f-bfc3-87059b88ab8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.620895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb5fa3-12e2-4a9f-bfc3-87059b88ab8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.622512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb5fa3-12e2-4a9f-bfc3-87059b88ab8e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.633601 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1362/2723] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:39.637796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.648559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f321ea-484e-4966-a3b4-1bb86ae7664a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.653304 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.653947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.654297 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.654568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] PATH_INFO: `/v3/projects/875a746f11504b00a1e229efa4e80655/users/bb36a99463c047c4a99be4b0889bfffa/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.668910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875a746f11504b00a1e229efa4e80655, user_id=bb36a99463c047c4a99be4b0889bfffa, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.672330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.685119 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1362/2724] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:39 2022] PUT /identity/v3/projects/875a746f11504b00a1e229efa4e80655/users/bb36a99463c047c4a99be4b0889bfffa/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:39.695595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.695913 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.696188 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.724858 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] MFA Rules not processed for user `bb36a99463c047c4a99be4b0889bfffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:39.726103 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:39.790401 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1363/2725] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:39.794933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.857043 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af098a-a7e3-45e9-9c27-ed7e0a733689 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.863630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.864244 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.864556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.864824 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.865481 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.867369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.894259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1363/2726] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:39.898656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:39.910017 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109cf2-d1e0-4d03-a2b3-b3c92b27de82 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:39.967743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:39.968388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:39.968673 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:39.968946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:39.969593 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:39.971525 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:39.993465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1364/2727] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:39 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:39.997500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:40.008610 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca69f-4401-4629-86a0-64fd483f299c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:40.013303 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:40.014065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.014350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.014628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.015296 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:40.016943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:40.028153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1364/2728] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:40.033012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:40.043209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96b08-18eb-44db-9e4c-d44c311b33a1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:40.048407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:40.049067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.049353 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.049633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] PATH_INFO: `/v3/projects/d44e84d49c0d4b4aa85331eb820f8bb7/users/8c666911824644c28975a760dd94b8a1/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.065268 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44e84d49c0d4b4aa85331eb820f8bb7, user_id=8c666911824644c28975a760dd94b8a1, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:40.067928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:40.083032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1365/2729] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:40 2022] PUT /identity/v3/projects/d44e84d49c0d4b4aa85331eb820f8bb7/users/8c666911824644c28975a760dd94b8a1/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:40.086765 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:40.098501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb688e-14c2-48be-a174-bdb8a2e6276e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:40.154901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc7b9b-cb83-4362-99bc-e65e17ef1d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:40.155571 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7b9b-cb83-4362-99bc-e65e17ef1d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.155850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7b9b-cb83-4362-99bc-e65e17ef1d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.156115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7b9b-cb83-4362-99bc-e65e17ef1d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.156777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc7b9b-cb83-4362-99bc-e65e17ef1d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:40.158377 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc7b9b-cb83-4362-99bc-e65e17ef1d08 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:40.168072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1365/2730] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:40.173040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:40.183248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a385-5668-4ab0-b54a-50729a7abf5f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:40.187841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:40.188521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.188803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.189074 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] PATH_INFO: `/v3/projects/d44e84d49c0d4b4aa85331eb820f8bb7/users/8c666911824644c28975a760dd94b8a1/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.190961 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44e84d49c0d4b4aa85331eb820f8bb7, user_id=8c666911824644c28975a760dd94b8a1, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:40.193626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:40.203779 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1366/2731] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:40 2022] PUT /identity/v3/projects/d44e84d49c0d4b4aa85331eb820f8bb7/users/8c666911824644c28975a760dd94b8a1/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:40.214747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.215040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.215315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.243375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] MFA Rules not processed for user `8c666911824644c28975a760dd94b8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:40.244619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:40.265205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:40.314260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1366/2732] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:40.324938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:40.327625 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:40.334338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:40.385864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02f589-863e-4c36-8932-405bc556f851 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:40.390105 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:40.390708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.390986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.391254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.400602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:40.402024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:40.411429 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1367/2733] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:40.457156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:40.458562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:40.470793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13abd3b-db24-40dd-a763-04a3549833f9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:40.474429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564e3b0-f9e2-4e24-af42-9eda241009d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:40.475031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564e3b0-f9e2-4e24-af42-9eda241009d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:40.475314 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564e3b0-f9e2-4e24-af42-9eda241009d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:40.475591 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564e3b0-f9e2-4e24-af42-9eda241009d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:40.531503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564e3b0-f9e2-4e24-af42-9eda241009d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:40.532981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564e3b0-f9e2-4e24-af42-9eda241009d4 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:40.541707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1367/2734] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:43.846665 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:43.909242 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dcd91-dcc5-4907-b6d3-c7561913c2d5 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:43.913753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:43.914379 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:43.914661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:43.914936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] PATH_INFO: `/v3/users/bb36a99463c047c4a99be4b0889bfffa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:43.916040 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb36a99463c047c4a99be4b0889bfffa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:43.917977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:43.962783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb36a99463c047c4a99be4b0889bfffa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:43.975569 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb36a99463c047c4a99be4b0889bfffa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:43.982444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb36a99463c047c4a99be4b0889bfffa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:43.989817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb36a99463c047c4a99be4b0889bfffa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:43.990632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb36a99463c047c4a99be4b0889bfffa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:43.996707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1368/2735] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:43 2022] DELETE /identity/v3/users/bb36a99463c047c4a99be4b0889bfffa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:44.001357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:44.001844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:44.001944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:44.020955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:44.022423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:44.083315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1368/2736] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:44.093570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:44.095204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:44.127368 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:44.156013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2a96-2e7e-41b7-80a8-bc5ace150099 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:44.159595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:44.160195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:44.160510 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:44.160783 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:44.169603 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:44.171613 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:44.180863 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1369/2737] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:44.450060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:44.451628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:44.464497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ec227-7d97-4e87-915e-bb3576adf342 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:44.470037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:44.470680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:44.470989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:44.471262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] PATH_INFO: `/v3/projects/875a746f11504b00a1e229efa4e80655` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:44.473295 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=875a746f11504b00a1e229efa4e80655)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:44.475161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:44.520826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '875a746f11504b00a1e229efa4e80655', 'request_id': 'req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.522960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:44.528094 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '875a746f11504b00a1e229efa4e80655', 'request_id': 'req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.534478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '875a746f11504b00a1e229efa4e80655', 'request_id': 'req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.548436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] The token cache is being invalidate because project 875a746f11504b00a1e229efa4e80655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:44.548736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.549513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.551057 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1369/2738] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:44 2022] DELETE /identity/v3/projects/875a746f11504b00a1e229efa4e80655 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:44.554997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:44.565366 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322a94d-0fe4-43b9-9851-dd2032c2f6cf service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:44.628847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:44.629422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:44.629700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:44.629968 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] PATH_INFO: `/v3/users/8c666911824644c28975a760dd94b8a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:44.631082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c666911824644c28975a760dd94b8a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:44.633016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:44.678038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c666911824644c28975a760dd94b8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.689794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c666911824644c28975a760dd94b8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.696682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c666911824644c28975a760dd94b8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.705290 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c666911824644c28975a760dd94b8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.705906 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-76f9f0f9-4450-45a8-a11d-f1561a82ba78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c666911824644c28975a760dd94b8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:44.711815 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1370/2739] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:44 2022] DELETE /identity/v3/users/8c666911824644c28975a760dd94b8a1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:44.985635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:44.987181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:45.002048 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036fca1-5609-4a2f-9cdd-f0cbfb8a2ba9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.058192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.058826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.059137 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.059411 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] PATH_INFO: `/v3/projects/d44e84d49c0d4b4aa85331eb820f8bb7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.060641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44e84d49c0d4b4aa85331eb820f8bb7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.063246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.111658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44e84d49c0d4b4aa85331eb820f8bb7', 'request_id': 'req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:45.117131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44e84d49c0d4b4aa85331eb820f8bb7', 'request_id': 'req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:45.123198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44e84d49c0d4b4aa85331eb820f8bb7', 'request_id': 'req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:45.135004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] The token cache is being invalidate because project d44e84d49c0d4b4aa85331eb820f8bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:45.135292 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:45.135909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c91b2b3f-5f78-4de1-9251-7c64e1e5fcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:45.137416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1370/2740] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:44 2022] DELETE /identity/v3/projects/d44e84d49c0d4b4aa85331eb820f8bb7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:45.142808 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.143328 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.143455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.163707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:45.164880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:45.207817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:45.224345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1371/2741] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:45.231970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.232346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.232625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.253276 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:45.254463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:45.262372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1371/2742] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:09:45.267269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.277120 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c0ebf-f5ee-4bfe-8eea-566ab739d843 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.282436 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.283024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.283301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.283592 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.284223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:45.284563 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.286127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.297666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1372/2743] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:45.301745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.310757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb7d07-38db-44ef-86d2-4786b8e31b8e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.315599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.316189 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.316503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.316771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.317398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.319300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.347008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1372/2744] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:45.351012 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.360817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ca0f1-7c65-4457-bced-5e9dffb7ab54 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.416295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.416832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.417115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.417386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.418009 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.419904 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.440326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1373/2745] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:45.444426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.455644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed014b0a-9e63-49bc-ba05-cec830730d59 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.460134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.460755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.461033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.461302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.461930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.463499 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.474115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1373/2746] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:45.477925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.487785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037e261-3ca8-4198-8e07-7eac585a4ea2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.493140 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d5665-7d75-438b-8e86-aec04235c0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.493724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5665-7d75-438b-8e86-aec04235c0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.494000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5665-7d75-438b-8e86-aec04235c0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.494271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d5665-7d75-438b-8e86-aec04235c0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.494891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d5665-7d75-438b-8e86-aec04235c0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.496506 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d5665-7d75-438b-8e86-aec04235c0bd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.507111 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1374/2747] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:45.511017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.522848 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fcc0e-8e63-4a86-ae8b-d8ca241ff92c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.528070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.528812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.529065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.529338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] PATH_INFO: `/v3/projects/cf31da14bb954daf8bfa30f45e5e52d3/users/593f60f2b3f340c48829633b9405e402/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.545167 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf31da14bb954daf8bfa30f45e5e52d3, user_id=593f60f2b3f340c48829633b9405e402, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.548131 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.560349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1374/2748] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:45 2022] PUT /identity/v3/projects/cf31da14bb954daf8bfa30f45e5e52d3/users/593f60f2b3f340c48829633b9405e402/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:45.570354 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.570663 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.570958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.599162 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] MFA Rules not processed for user `593f60f2b3f340c48829633b9405e402`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:45.600386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:45.667265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1375/2749] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:45.672519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.681140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:45.735390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93d597-f4f1-4c2b-a68d-a8537650e411 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.740022 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.740659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.740943 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.741218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.741844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.743793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.770695 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1375/2750] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:45.774999 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.786655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e7547-749c-4091-97d5-674caa56d7fa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.828351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:45.844591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.845205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.845491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.845803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.846442 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.848378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.869394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1376/2751] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:45 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:45.873380 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.885644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce11b-e59a-44ea-b292-48354867feab admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.890386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.891008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.891290 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.891564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.892207 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.893900 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.906218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1376/2752] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:45.910917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.922704 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ba31d-83c4-40aa-890b-aef076b90cee admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:45.927959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:45.928660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:45.928947 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:45.929226 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] PATH_INFO: `/v3/projects/7080a8a5c5954b9389429b53d21fd200/users/7b77cfd2a9dc4acf8cf98fc91fd25051/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:45.944873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7080a8a5c5954b9389429b53d21fd200, user_id=7b77cfd2a9dc4acf8cf98fc91fd25051, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:45.947818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:45.962821 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1377/2753] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:45 2022] PUT /identity/v3/projects/7080a8a5c5954b9389429b53d21fd200/users/7b77cfd2a9dc4acf8cf98fc91fd25051/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:45.966904 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:45.977956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623dfe10-0585-4728-85d7-30424112adde admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:46.044194 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b036f1f-bdc7-4bd5-b12d-8bfd7501f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:46.046740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b036f1f-bdc7-4bd5-b12d-8bfd7501f5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:46.047009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b036f1f-bdc7-4bd5-b12d-8bfd7501f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:46.047337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b036f1f-bdc7-4bd5-b12d-8bfd7501f5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:46.047945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b036f1f-bdc7-4bd5-b12d-8bfd7501f5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:46.049684 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b036f1f-bdc7-4bd5-b12d-8bfd7501f5ef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:46.060243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1377/2754] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:46.065251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:46.077434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387ee03-a4f8-4b57-bc86-a3ddac94d710 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:46.090085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:46.090776 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:46.091067 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:46.091347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] PATH_INFO: `/v3/projects/7080a8a5c5954b9389429b53d21fd200/users/7b77cfd2a9dc4acf8cf98fc91fd25051/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:46.093570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7080a8a5c5954b9389429b53d21fd200, user_id=7b77cfd2a9dc4acf8cf98fc91fd25051, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:46.096486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:46.108631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1378/2755] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:46 2022] PUT /identity/v3/projects/7080a8a5c5954b9389429b53d21fd200/users/7b77cfd2a9dc4acf8cf98fc91fd25051/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:46.119845 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:46.120136 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:46.120452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:46.150520 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] MFA Rules not processed for user `7b77cfd2a9dc4acf8cf98fc91fd25051`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:46.151736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:46.220441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1378/2756] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:46.231575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:46.234781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:46.294363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a205f8-5fc3-430c-ab52-8a68dbdeb5f0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:46.298912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3bd9-b74e-4a65-8e9c-ce103c967621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:46.299513 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3bd9-b74e-4a65-8e9c-ce103c967621 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:46.299788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3bd9-b74e-4a65-8e9c-ce103c967621 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:46.300057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3bd9-b74e-4a65-8e9c-ce103c967621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:46.308958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3bd9-b74e-4a65-8e9c-ce103c967621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:46.310355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3bd9-b74e-4a65-8e9c-ce103c967621 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:46.320660 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1379/2757] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:46.422596 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:46.427256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:46.433549 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df81cd-de32-46c5-9cc9-463cff3ce70f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:46.437238 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:46.437828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:46.438108 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:46.438379 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:46.494050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:46.495398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:46.504042 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1379/2758] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:49.166482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:49.166998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:49.167104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:49.218597 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:49.219816 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:49.287584 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1380/2759] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:09:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:49.330206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:49.331666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:49.427463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab174eff-ec5d-41c9-8ccd-cbaebcee9742 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:49.434813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01a7bf-4af2-4056-8093-62637e43053d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:49.435787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01a7bf-4af2-4056-8093-62637e43053d service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:49.436312 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01a7bf-4af2-4056-8093-62637e43053d service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:49.436602 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01a7bf-4af2-4056-8093-62637e43053d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:49.446740 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01a7bf-4af2-4056-8093-62637e43053d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:49.448359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01a7bf-4af2-4056-8093-62637e43053d service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:49.458173 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1380/2760] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:09:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:09:51.349331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:51.376697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:51.435678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d624-ec2c-42ae-8f0a-c1ab7f7c2dd5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:51.441659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:51.442428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:51.442713 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:51.442985 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] PATH_INFO: `/v3/users/593f60f2b3f340c48829633b9405e402` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:51.444332 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=593f60f2b3f340c48829633b9405e402)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:51.450059 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:51.523849 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '593f60f2b3f340c48829633b9405e402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:51.549153 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '593f60f2b3f340c48829633b9405e402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:51.557785 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '593f60f2b3f340c48829633b9405e402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:51.580761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '593f60f2b3f340c48829633b9405e402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:51.582717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '593f60f2b3f340c48829633b9405e402'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:51.595150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1381/2761] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:51 2022] DELETE /identity/v3/users/593f60f2b3f340c48829633b9405e402 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:51.609934 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:51.610888 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:51.611344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:51.640608 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:51.641953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:51.747581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1381/2762] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:51.755038 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:51.756972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:51.839314 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47afd85b-1cfd-4ad8-946f-6412700a7d94 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:51.844190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:51.845115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:51.845435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:51.845744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:51.856664 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:51.858890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:51.868896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1382/2763] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:52.176784 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:52.178400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:52.205774 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2406627-1cb3-4bb9-9438-25f4a0c707a8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:52.211478 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:52.212313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:52.213070 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:52.214820 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] PATH_INFO: `/v3/projects/cf31da14bb954daf8bfa30f45e5e52d3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:52.221051 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf31da14bb954daf8bfa30f45e5e52d3)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:52.223004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:52.232090 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:52.276559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf31da14bb954daf8bfa30f45e5e52d3', 'request_id': 'req-f4e7aa0f-4752-4081-ad1e-95125d1cc609'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.283743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf31da14bb954daf8bfa30f45e5e52d3', 'request_id': 'req-f4e7aa0f-4752-4081-ad1e-95125d1cc609'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.290311 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf31da14bb954daf8bfa30f45e5e52d3', 'request_id': 'req-f4e7aa0f-4752-4081-ad1e-95125d1cc609'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.305425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] The token cache is being invalidate because project cf31da14bb954daf8bfa30f45e5e52d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:52.305745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.306358 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.307851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1382/2764] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:52 2022] DELETE /identity/v3/projects/cf31da14bb954daf8bfa30f45e5e52d3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:52.311975 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:52.321962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc83a-1a94-4a93-a9db-0941e5229109 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:52.380752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:52.439651 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:52.440893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:52.440893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:52.441116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] PATH_INFO: `/v3/users/7b77cfd2a9dc4acf8cf98fc91fd25051` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:52.442407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b77cfd2a9dc4acf8cf98fc91fd25051)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:52.444688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:52.564658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b77cfd2a9dc4acf8cf98fc91fd25051'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.580944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b77cfd2a9dc4acf8cf98fc91fd25051'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.589564 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b77cfd2a9dc4acf8cf98fc91fd25051'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.599990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b77cfd2a9dc4acf8cf98fc91fd25051'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.601698 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-66484f03-73a2-44ed-9f46-30b22d44a0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b77cfd2a9dc4acf8cf98fc91fd25051'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:52.609403 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1383/2765] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:52 2022] DELETE /identity/v3/users/7b77cfd2a9dc4acf8cf98fc91fd25051 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:52.938352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:52.941534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:52.975007 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7aa0f-4752-4081-ad1e-95125d1cc609 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.048360 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.049156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.049440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.049861 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] PATH_INFO: `/v3/projects/7080a8a5c5954b9389429b53d21fd200` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.051040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7080a8a5c5954b9389429b53d21fd200)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.053186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.153513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7080a8a5c5954b9389429b53d21fd200', 'request_id': 'req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:53.160736 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7080a8a5c5954b9389429b53d21fd200', 'request_id': 'req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:53.172196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7080a8a5c5954b9389429b53d21fd200', 'request_id': 'req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:53.192885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] The token cache is being invalidate because project 7080a8a5c5954b9389429b53d21fd200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:53.193315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:53.194400 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f84005dc-6eb4-488c-a77e-e0a64aeb1b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:53.199909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1383/2766] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:52 2022] DELETE /identity/v3/projects/7080a8a5c5954b9389429b53d21fd200 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:53.202587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.202998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.203349 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.237417 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:53.239301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:53.338494 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1384/2767] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:53.346325 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.346757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.347134 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.379352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:53.382800 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:53.397384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1384/2768] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:53.401474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:53.413599 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c7e78-f7a9-4ea0-9d92-153eb8d244bc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.441693 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.442327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.442614 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.442953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.443546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:53.443848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.445488 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.464195 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1385/2769] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:53.468880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:53.479559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba21104-e0ae-44f8-9175-67ab4082e946 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.492247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.492914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.493294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.493668 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.494417 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.496599 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.518214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:53.537999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1385/2770] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:53 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:53.541905 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:53.555981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17f966-9293-4820-bd7d-e0ce192df7e5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.605424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:53.662780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.663405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.663686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.663960 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.664642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.666562 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.693915 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1386/2771] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:53 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:53.697958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:53.709176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2e2ef-8b29-461f-bcfe-2c2a67745c45 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.713947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.714654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.715053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.715429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.716161 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.718860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.729923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1386/2772] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:53.734829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:53.745386 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed7bb-3f41-4b9c-abba-c8dbc56e3d1e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.752437 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7bfef-9f89-4dc0-aa1f-853f4f32283e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.753058 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7bfef-9f89-4dc0-aa1f-853f4f32283e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.753357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7bfef-9f89-4dc0-aa1f-853f4f32283e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.753648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7bfef-9f89-4dc0-aa1f-853f4f32283e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.754293 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7bfef-9f89-4dc0-aa1f-853f4f32283e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.755892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7bfef-9f89-4dc0-aa1f-853f4f32283e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.772265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1387/2773] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:53.777352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:53.789585 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952319d-bc0d-4cd4-87a7-3f7e850cce52 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:53.795609 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:53.796324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.796843 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.797152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] PATH_INFO: `/v3/projects/d1d434cf1a644d50965b0b3c250d5e6c/users/96dc5341a57b408080fe1d807e2dd202/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.813359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d434cf1a644d50965b0b3c250d5e6c, user_id=96dc5341a57b408080fe1d807e2dd202, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:53.816201 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:53.836356 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1387/2774] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:53 2022] PUT /identity/v3/projects/d1d434cf1a644d50965b0b3c250d5e6c/users/96dc5341a57b408080fe1d807e2dd202/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:53.846271 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:53.846685 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:53.847213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:53.883373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] MFA Rules not processed for user `96dc5341a57b408080fe1d807e2dd202`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:53.886285 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:53.977073 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1388/2775] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:53.981452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.111637 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b79db4-3a19-4b0f-bbf3-31b7c99f2642 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.123713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.125294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.125914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.127378 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.130537 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.136719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.178657 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1388/2776] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:53 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:54.183287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.196407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2342-5843-41d5-886f-8d5b960ed7a9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.285376 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.286002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.286287 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.286568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.287209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.289591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.315363 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1389/2777] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:54 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:54.319348 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.330320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e232f-dd68-4fbe-869a-240da83d600b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.335407 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.336088 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.336471 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.336842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.337598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.339274 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.352334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1389/2778] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:54.356261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.369010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db5c35-21b6-4c17-9b44-c3b939a96186 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.375144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae06b0-c99c-4110-a3b7-5cafedfdf733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.375848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae06b0-c99c-4110-a3b7-5cafedfdf733 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.376206 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae06b0-c99c-4110-a3b7-5cafedfdf733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.376617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae06b0-c99c-4110-a3b7-5cafedfdf733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.377471 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae06b0-c99c-4110-a3b7-5cafedfdf733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.379178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae06b0-c99c-4110-a3b7-5cafedfdf733 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.393746 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1390/2779] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:54.394867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.406054 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45ea13-c4dd-467d-884f-0015ca82e929 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.410766 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.411414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.411694 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.411970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] PATH_INFO: `/v3/projects/23b5086381194aa7943fd8451aabfc5c/users/0e56a27131f04a5683206dab41aab7b4/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.429648 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b5086381194aa7943fd8451aabfc5c, user_id=0e56a27131f04a5683206dab41aab7b4, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.432456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.447612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1390/2780] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:54 2022] PUT /identity/v3/projects/23b5086381194aa7943fd8451aabfc5c/users/0e56a27131f04a5683206dab41aab7b4/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:09:54.457879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.458418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.458579 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.490240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] MFA Rules not processed for user `0e56a27131f04a5683206dab41aab7b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:54.491574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:54.506107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:54.568197 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1391/2781] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:54.572372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.640583 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a92d5-8c39-4b09-b899-47c4472eb6db admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.647632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.648536 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.648958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.649242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.649915 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.651925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.668203 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:54.689240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.692567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1391/2782] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:54 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:54.692724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:54.693623 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.713524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeeab4-ec81-49b2-9445-8b051b875fe3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.776319 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49bffc-64ac-4261-ae8a-6742acd32542 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.782009 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.782644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.782925 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.783269 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.783909 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.785877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.797497 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.798145 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.798422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.798690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] PATH_INFO: `/v3/users/d166424a556e47e3b8ddee852d10b657` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.799856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d166424a556e47e3b8ddee852d10b657)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.802435 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.813211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1392/2783] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:54 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:54.817306 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.829519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa318f-5cab-4de5-825e-b9d0f477a35b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.836157 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.837432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.838010 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.838505 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.839320 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.841576 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.856704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1393/2784] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:54.862700 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.869814 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd166424a556e47e3b8ddee852d10b657'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:54.874300 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3c72d-2e53-440f-beef-d2a01492b37f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:54.879153 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:54.879819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.880102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.880416 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] PATH_INFO: `/v3/projects/9d74e40a797a48328d1e02ba0c9d83aa/users/ed27b2e7280142a1af7927f952918baa/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.888126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd166424a556e47e3b8ddee852d10b657'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:54.897769 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd166424a556e47e3b8ddee852d10b657'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:54.901080 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d74e40a797a48328d1e02ba0c9d83aa, user_id=ed27b2e7280142a1af7927f952918baa, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:54.904063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:54.908670 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd166424a556e47e3b8ddee852d10b657'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:54.909382 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8a1eaf7c-732a-4e18-a655-91b3c3ebdad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd166424a556e47e3b8ddee852d10b657'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:54.919752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1392/2785] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:09:54 2022] DELETE /identity/v3/users/d166424a556e47e3b8ddee852d10b657 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:54.920235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1394/2786] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:54 2022] PUT /identity/v3/projects/9d74e40a797a48328d1e02ba0c9d83aa/users/ed27b2e7280142a1af7927f952918baa/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:54.921970 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:54.922259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:54.922536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:54.924242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:54.926079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:54.950446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:54.952259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:54.997731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:55.004226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c9c12-f6b7-416a-a9a2-f8aca7da8f49 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:55.036719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1393/2787] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:55.046971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:55.048626 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:55.089448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:55.098261 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:55.098885 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.099169 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.099441 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.100147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:55.105402 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:55.125660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1395/2788] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:09:55.133635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:55.142885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9c9ae-39e3-4d9a-b467-95ad1586de3a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:55.147538 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8716ed-e6c0-4e96-99db-e207dbd244e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:55.148326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8716ed-e6c0-4e96-99db-e207dbd244e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.148619 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8716ed-e6c0-4e96-99db-e207dbd244e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.149045 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8716ed-e6c0-4e96-99db-e207dbd244e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.152837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff33b2f-ed04-437d-836d-2451d744b6ab admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:55.159199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:55.160073 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.160458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.160595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8716ed-e6c0-4e96-99db-e207dbd244e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:55.160862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] PATH_INFO: `/v3/projects/9d74e40a797a48328d1e02ba0c9d83aa/users/ed27b2e7280142a1af7927f952918baa/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.162121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8716ed-e6c0-4e96-99db-e207dbd244e3 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:55.163000 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d74e40a797a48328d1e02ba0c9d83aa, user_id=ed27b2e7280142a1af7927f952918baa, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:55.165745 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:55.173977 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1394/2789] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:09:55.205523 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1396/2790] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:55 2022] PUT /identity/v3/projects/9d74e40a797a48328d1e02ba0c9d83aa/users/ed27b2e7280142a1af7927f952918baa/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:55.207682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.207986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.208261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.250317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] MFA Rules not processed for user `ed27b2e7280142a1af7927f952918baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:55.252470 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:55.403261 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1395/2791] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:55.414924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:55.416692 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:55.490448 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994981a-8e6f-4956-91df-93441ab2dd0f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:55.494562 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:55.495391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.495775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.496151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.498062 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:55.560203 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:55.563446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:55.599946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:55.602082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d767a-9773-4cdb-a812-7bcc1f0f34fd service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:55.626862 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1397/2792] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:55.719327 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cabfdb3-803d-473e-9270-8f12287486c7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:55.724448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:55.725089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.725369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.725636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] PATH_INFO: `/v3/projects/f3a4a0ea38d64717b1579c1362aa3d0a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.727165 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a4a0ea38d64717b1579c1362aa3d0a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:55.732503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:55.762574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:55.828917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a4a0ea38d64717b1579c1362aa3d0a', 'request_id': 'req-488bd03a-bbbb-472f-8da9-0969393e3872'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:55.840531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a4a0ea38d64717b1579c1362aa3d0a', 'request_id': 'req-488bd03a-bbbb-472f-8da9-0969393e3872'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:55.856447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a4a0ea38d64717b1579c1362aa3d0a', 'request_id': 'req-488bd03a-bbbb-472f-8da9-0969393e3872'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:55.883518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] The token cache is being invalidate because project f3a4a0ea38d64717b1579c1362aa3d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:09:55.884633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:55.885355 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-488bd03a-bbbb-472f-8da9-0969393e3872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:09:55.887098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1396/2793] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:09:55 2022] DELETE /identity/v3/projects/f3a4a0ea38d64717b1579c1362aa3d0a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:55.893452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:55.893804 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:55.894081 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:55.925240 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:55.929832 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:56.025670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1398/2794] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:56.035817 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d66153-2617-4774-82ce-710f8d10560c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.036108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d66153-2617-4774-82ce-710f8d10560c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.036414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d66153-2617-4774-82ce-710f8d10560c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.069458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81d66153-2617-4774-82ce-710f8d10560c None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:56.070879 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81d66153-2617-4774-82ce-710f8d10560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:56.097761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.103577 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1397/2795] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:56.106091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1afe7a-bf63-467f-9147-48ce439c51de None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.111323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.111980 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.112439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.112716 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.113361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:09:56.113661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.115322 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.136409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1399/2796] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:09:56.141714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d66153-2617-4774-82ce-710f8d10560c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.156995 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d66153-2617-4774-82ce-710f8d10560c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.162833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.163648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.164001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.164731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.165496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.167547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.216380 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1398/2797] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:56 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:56.217807 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.229715 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec410d5-9800-4c2c-a7dd-4218259cdbf3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.253542 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:56.300761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.301369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.301649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.301918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.302543 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.304658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.335342 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1400/2798] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:56 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:56.337446 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.352068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634c344-d4d4-4c70-a946-71c0b05358ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.359819 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.360467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.360748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.361016 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.361649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.363225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.373688 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1399/2799] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:56.377948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.387811 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844103c8-4095-4d61-be8f-2d0db21fad03 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.392717 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7b493-08a9-4d6c-a54f-cc775a76eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.393309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7b493-08a9-4d6c-a54f-cc775a76eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.393612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7b493-08a9-4d6c-a54f-cc775a76eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.393880 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7b493-08a9-4d6c-a54f-cc775a76eb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.394515 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7b493-08a9-4d6c-a54f-cc775a76eb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.396181 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7b493-08a9-4d6c-a54f-cc775a76eb0a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.407005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1401/2800] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:56.410880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.421282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39683067-48e7-4d3d-b778-4a39f82802eb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.425765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.426398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.426678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.426950 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] PATH_INFO: `/v3/projects/3b0ff409a0dd4e449e76998fb6591656/users/c945d160a0694309af20a850e989541e/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.441743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0ff409a0dd4e449e76998fb6591656, user_id=c945d160a0694309af20a850e989541e, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.445245 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.464598 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1400/2801] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:56 2022] PUT /identity/v3/projects/3b0ff409a0dd4e449e76998fb6591656/users/c945d160a0694309af20a850e989541e/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:56.475245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.475540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.475877 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.630706 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] MFA Rules not processed for user `c945d160a0694309af20a850e989541e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:56.632055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:56.719152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1402/2802] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:56.723682 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.798804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876699c6-173d-4738-8e5d-2ce4decab992 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.804136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.804801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.805086 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.805357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.806000 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.808046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.839398 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1401/2803] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:56 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:56.846259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.862508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f47bfc-665d-4d1c-a364-366793c7e3b4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.934532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.935141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.935420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.935719 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.936371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.938275 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:56.963436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1403/2804] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:56 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:56.967946 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:56.980810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da45f69-b7f7-4bde-88b1-e57aa6237a4b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:56.987204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:56.987851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:56.988155 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:56.988557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:56.989873 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:56.991453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.008433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1402/2805] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:57.012704 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.027484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df3db3-d178-4e98-95aa-331586dcb155 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.032809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55620b2-801e-4e2c-a513-dc1ba8e9e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.033429 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55620b2-801e-4e2c-a513-dc1ba8e9e85d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.033708 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55620b2-801e-4e2c-a513-dc1ba8e9e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.033988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55620b2-801e-4e2c-a513-dc1ba8e9e85d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.034780 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55620b2-801e-4e2c-a513-dc1ba8e9e85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.036597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55620b2-801e-4e2c-a513-dc1ba8e9e85d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.051494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1404/2806] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:57.052912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.062205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:57.068524 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210df49-762d-48f7-bee9-b788fd8051ce admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.077847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.079006 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.085547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.085547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] PATH_INFO: `/v3/projects/d7c44120fdff48839380322afb5fbaa6/users/d31dcaa2f83e4118ab9bb989ab03e3c2/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.109628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c44120fdff48839380322afb5fbaa6, user_id=d31dcaa2f83e4118ab9bb989ab03e3c2, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.112575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.129582 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1403/2807] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:57 2022] PUT /identity/v3/projects/d7c44120fdff48839380322afb5fbaa6/users/d31dcaa2f83e4118ab9bb989ab03e3c2/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:57.140822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.141444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.141924 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.164850 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:57.177808 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] MFA Rules not processed for user `d31dcaa2f83e4118ab9bb989ab03e3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:57.179192 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:57.269307 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1405/2808] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:09:57.274483 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.392320 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c6d06-e85c-4f49-abed-50241b0c7667 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.398157 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.398893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.399269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.399635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.400479 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.402796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.459240 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1404/2809] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:09:57 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:09:57.460906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.477656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.481416 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:57.490699 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686d1ad-22c0-4572-b5cd-9dddd71d4169 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.562047 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267dba-634d-467b-b3ab-896160488ed1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.566499 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.567352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.567798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.568264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.570077 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:57.597502 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.598680 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.599539 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.599813 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.600487 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.602884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.650116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.651707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.656641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1406/2810] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:09:57 2022] POST /identity/v3/users => generated 299 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:09:57.657757 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.663548 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1405/2811] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:57.674687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e41b4e-23f3-47cc-a589-a00aa6a9f076 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.680799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.681558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.681846 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.683647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.683647 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.684826 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.719660 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1407/2812] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:57.723864 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.742678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb092e6-bd4d-4e56-bba0-ea57fe9495db service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.750146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.750965 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.751518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.751765 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] PATH_INFO: `/v3/projects/65b203e6701147139a1ad61e7a63000f/users/f1ee1bbf4cbc47dfb458c808995ccb59/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.782422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b203e6701147139a1ad61e7a63000f, user_id=f1ee1bbf4cbc47dfb458c808995ccb59, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.786718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.790473 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:57.810394 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1406/2813] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:57 2022] PUT /identity/v3/projects/65b203e6701147139a1ad61e7a63000f/users/f1ee1bbf4cbc47dfb458c808995ccb59/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:57.814563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.829447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5a6de-7c1e-44f5-8786-dab10cc5d759 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.917616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7182d4b-b035-43a0-9dd7-4931451d52f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.918384 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7182d4b-b035-43a0-9dd7-4931451d52f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.918698 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7182d4b-b035-43a0-9dd7-4931451d52f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.919034 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7182d4b-b035-43a0-9dd7-4931451d52f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.919762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7182d4b-b035-43a0-9dd7-4931451d52f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.922452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7182d4b-b035-43a0-9dd7-4931451d52f5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:57.939553 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1408/2814] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:09:57.941558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:57.955216 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb0006-e273-4743-80e0-9e5e98b71779 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:57.971516 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:57.972997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:57.973264 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:57.973570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] PATH_INFO: `/v3/projects/65b203e6701147139a1ad61e7a63000f/users/f1ee1bbf4cbc47dfb458c808995ccb59/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:57.975813 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b203e6701147139a1ad61e7a63000f, user_id=f1ee1bbf4cbc47dfb458c808995ccb59, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:57.978680 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:58.000066 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1407/2815] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:09:57 2022] PUT /identity/v3/projects/65b203e6701147139a1ad61e7a63000f/users/f1ee1bbf4cbc47dfb458c808995ccb59/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:09:58.011739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:58.012032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:58.012498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:58.049937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:09:58.057494 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] MFA Rules not processed for user `f1ee1bbf4cbc47dfb458c808995ccb59`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:09:58.058459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:58.171459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1409/2816] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:09:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:09:58.183616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:58.185395 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:58.272799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c91df-50a8-4853-bfbb-05cd7320c81f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:58.286338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:58.286932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:58.287218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:58.287491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:58.361177 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:58.362850 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:58.384486 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1408/2817] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:09:58.444892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:09:58.456883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe7fb2-1987-4671-8820-9307cc357506 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:09:58.464122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:09:58.465041 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:09:58.465669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:09:58.466101 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:09:58.469512 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:09:58.567902 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:09:58.569669 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:09:58.585500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1410/2818] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:09:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:00.008762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:00.021117 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001369-a0ac-42b3-ade6-b2b13a95c2b4 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:00.025781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:00.026649 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:00.027109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:00.027703 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:00.161032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:00.162773 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:00.174202 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1409/2819] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:00.825750 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:00.834906 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4852-5235-4a3d-a540-ca7060257b68 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:00.838595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a79cc-bc06-4b2a-b2d9-53532086a299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:00.839186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a79cc-bc06-4b2a-b2d9-53532086a299 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:00.839459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a79cc-bc06-4b2a-b2d9-53532086a299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:00.839724 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a79cc-bc06-4b2a-b2d9-53532086a299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:00.855044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a79cc-bc06-4b2a-b2d9-53532086a299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:00.857242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a79cc-bc06-4b2a-b2d9-53532086a299 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:00.869227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1411/2820] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:02.716434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:02.720678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:02.772238 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:02.809860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea180070-8806-42d0-9517-1a9859e04e5e service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:02.815168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:02.815833 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:02.816112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:02.816415 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] PATH_INFO: `/v3/users/96dc5341a57b408080fe1d807e2dd202` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:02.817655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96dc5341a57b408080fe1d807e2dd202)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:02.819615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:02.882419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96dc5341a57b408080fe1d807e2dd202'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:02.900427 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96dc5341a57b408080fe1d807e2dd202'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:02.932001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96dc5341a57b408080fe1d807e2dd202'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:02.941615 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96dc5341a57b408080fe1d807e2dd202'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:02.942262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96dc5341a57b408080fe1d807e2dd202'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:02.948722 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1410/2821] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:02 2022] DELETE /identity/v3/users/96dc5341a57b408080fe1d807e2dd202 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:02.953367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:02.953882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:02.953882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:02.974734 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:02.976186 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:03.047032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1412/2822] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:03.059791 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:03.061347 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:03.139724 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b99f-60b0-41c1-acd6-afeb04b3cf33 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:03.144263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:03.145131 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:03.145504 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:03.145848 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:03.161981 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:03.163752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:03.176378 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1411/2823] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:03.508315 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:03.509777 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:03.548162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008da154-6231-48c2-b0c3-a35068fcc3c9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:03.568514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:03.568514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:03.568514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:03.568514 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] PATH_INFO: `/v3/projects/d1d434cf1a644d50965b0b3c250d5e6c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:03.568910 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d434cf1a644d50965b0b3c250d5e6c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:03.569450 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:03.587459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:03.627743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d434cf1a644d50965b0b3c250d5e6c', 'request_id': 'req-a6cd0114-8857-4620-854c-5b88f76c0a78'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.635869 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d434cf1a644d50965b0b3c250d5e6c', 'request_id': 'req-a6cd0114-8857-4620-854c-5b88f76c0a78'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.643031 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d434cf1a644d50965b0b3c250d5e6c', 'request_id': 'req-a6cd0114-8857-4620-854c-5b88f76c0a78'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.659666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] The token cache is being invalidate because project d1d434cf1a644d50965b0b3c250d5e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:03.659898 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.660593 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.665873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1413/2824] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:03 2022] DELETE /identity/v3/projects/d1d434cf1a644d50965b0b3c250d5e6c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:03.666949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:03.683288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460ef7d-6208-4243-8253-35105b8c5ee6 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:03.769041 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:03.781430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:03.782318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:03.782790 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:03.783062 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] PATH_INFO: `/v3/users/0e56a27131f04a5683206dab41aab7b4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:03.784215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e56a27131f04a5683206dab41aab7b4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:03.786228 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:03.855124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e56a27131f04a5683206dab41aab7b4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.876283 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e56a27131f04a5683206dab41aab7b4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.884532 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e56a27131f04a5683206dab41aab7b4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.897452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e56a27131f04a5683206dab41aab7b4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.898190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e56a27131f04a5683206dab41aab7b4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:03.908173 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1412/2825] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:03 2022] DELETE /identity/v3/users/0e56a27131f04a5683206dab41aab7b4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:04.275555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:04.277619 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:04.295462 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd0114-8857-4620-854c-5b88f76c0a78 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:04.369540 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:04.370084 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:04.370362 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:04.370630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] PATH_INFO: `/v3/projects/23b5086381194aa7943fd8451aabfc5c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:04.371743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b5086381194aa7943fd8451aabfc5c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:04.374685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:04.428533 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b5086381194aa7943fd8451aabfc5c', 'request_id': 'req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.437432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b5086381194aa7943fd8451aabfc5c', 'request_id': 'req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.444557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b5086381194aa7943fd8451aabfc5c', 'request_id': 'req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.461239 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] The token cache is being invalidate because project 23b5086381194aa7943fd8451aabfc5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:04.461615 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.462331 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.464068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1414/2826] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:04 2022] DELETE /identity/v3/projects/23b5086381194aa7943fd8451aabfc5c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:04.468891 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:04.470359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:04.489319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98121e-c1a5-4761-af21-2b98362e64e8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:04.581789 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:04.583091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:04.583350 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:04.583690 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] PATH_INFO: `/v3/users/ed27b2e7280142a1af7927f952918baa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:04.584861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed27b2e7280142a1af7927f952918baa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:04.586803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:04.653643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed27b2e7280142a1af7927f952918baa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.670221 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed27b2e7280142a1af7927f952918baa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.684223 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed27b2e7280142a1af7927f952918baa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.689158 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:04.697345 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed27b2e7280142a1af7927f952918baa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.698762 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed27b2e7280142a1af7927f952918baa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:04.712861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1413/2827] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:04 2022] DELETE /identity/v3/users/ed27b2e7280142a1af7927f952918baa => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:05.016458 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.017923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:05.034689 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4649d-0eea-44cf-9ae7-7d76c682bd31 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.070996 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:05.110725 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.111666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.111963 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.112359 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] PATH_INFO: `/v3/users/c945d160a0694309af20a850e989541e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.113901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c945d160a0694309af20a850e989541e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.114440 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.116091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:05.116214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.147178 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6359b9a-e366-480c-8573-1619da400f0c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.166700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.167284 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.167570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.167841 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] PATH_INFO: `/v3/projects/9d74e40a797a48328d1e02ba0c9d83aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.169450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d74e40a797a48328d1e02ba0c9d83aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.171487 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.220453 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c945d160a0694309af20a850e989541e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.231210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c945d160a0694309af20a850e989541e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.249196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c945d160a0694309af20a850e989541e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.250391 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c945d160a0694309af20a850e989541e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.254602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d74e40a797a48328d1e02ba0c9d83aa', 'request_id': 'req-c765aa40-1655-452d-8015-064c2e96fb72'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.261033 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ec2c1b67-a6fa-4e1b-bc07-e5292cd826b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c945d160a0694309af20a850e989541e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.264574 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d74e40a797a48328d1e02ba0c9d83aa', 'request_id': 'req-c765aa40-1655-452d-8015-064c2e96fb72'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.269425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1415/2828] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:05 2022] DELETE /identity/v3/users/c945d160a0694309af20a850e989541e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:05.276256 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.276932 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d74e40a797a48328d1e02ba0c9d83aa', 'request_id': 'req-c765aa40-1655-452d-8015-064c2e96fb72'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.277423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.277922 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.303774 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] The token cache is being invalidate because project 9d74e40a797a48328d1e02ba0c9d83aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:05.304402 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.305079 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:05.305482 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c765aa40-1655-452d-8015-064c2e96fb72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:05.306839 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:05.307844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1414/2829] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:05 2022] DELETE /identity/v3/projects/9d74e40a797a48328d1e02ba0c9d83aa => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:05.314875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf813a-524b-4ccc-8f06-aee8eb7b378a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.315312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf813a-524b-4ccc-8f06-aee8eb7b378a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.315741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf813a-524b-4ccc-8f06-aee8eb7b378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.374220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-79bf813a-524b-4ccc-8f06-aee8eb7b378a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:05.376312 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-79bf813a-524b-4ccc-8f06-aee8eb7b378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:05.402762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1416/2830] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:05.413728 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.415753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:05.466085 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1415/2831] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:05.505940 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9929e26-f523-439b-bb6d-b3894940d48b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.510286 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.511216 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.511573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.511897 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.523658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:05.525790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.527532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.535112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.535388 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.535803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.558666 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:05.559967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:05.561361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1417/2832] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:10:05.573928 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1416/2833] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:05.575818 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.598164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d886b3a-7f4a-4d13-8791-e21e1c41aaa6 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.609005 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.610191 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.610899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.611363 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.612164 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:10:05.612567 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.614516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.631454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1418/2834] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:10:05.634262 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.646291 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21fb5a-cea3-4a22-a2c8-e5c40c60a8da None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.651741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.652568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.652912 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.653301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.655480 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.657529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.700309 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1417/2835] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:10:05 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:05.705089 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.720944 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc569a56-9a84-4aaa-9183-cb6d5ab3c721 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.797558 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.798334 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.799285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.799735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.800381 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.802557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.834582 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1419/2836] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:10:05 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:05.840809 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.858596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64c076-a0d8-4d07-86a8-2809af5f40b6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.871295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.871949 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.872234 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.872536 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.873899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.875596 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.890422 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1418/2837] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:10:05.891809 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.914086 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4723e-1e63-4f90-a761-1ba836039f33 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.921828 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b40ae9-c834-43c4-8070-52ec44af7808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.922452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b40ae9-c834-43c4-8070-52ec44af7808 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.922814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b40ae9-c834-43c4-8070-52ec44af7808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.923091 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b40ae9-c834-43c4-8070-52ec44af7808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.923744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b40ae9-c834-43c4-8070-52ec44af7808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.925683 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b40ae9-c834-43c4-8070-52ec44af7808 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:05.947324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1420/2838] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:10:05.951803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:05.969635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff025ee-9df3-486b-8093-e7b1ea8474af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:05.975249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:05.975855 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:05.976179 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:05.976522 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] PATH_INFO: `/v3/projects/f3ecef1d1c9c40eaa48215e4000d6f5c/users/cb83494f073341019247c23545e801ff/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:05.994954 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ecef1d1c9c40eaa48215e4000d6f5c, user_id=cb83494f073341019247c23545e801ff, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:05.999372 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:06.031500 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1419/2839] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:10:05 2022] PUT /identity/v3/projects/f3ecef1d1c9c40eaa48215e4000d6f5c/users/cb83494f073341019247c23545e801ff/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:06.035114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:06.036218 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:06.037074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:06.105225 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] MFA Rules not processed for user `cb83494f073341019247c23545e801ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:06.106979 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.183899 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:06.188741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.240454 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:06.264321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1421/2840] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:06.297644 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:06.301526 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.312542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e6bbb-a8fe-4723-9b61-cd48c5c52bf1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:06.317972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:06.319082 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:06.319555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:06.319834 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] PATH_INFO: `/v3/projects/3b0ff409a0dd4e449e76998fb6591656` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:06.321094 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0ff409a0dd4e449e76998fb6591656)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:06.323039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:06.407288 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3763325-db7e-4065-a3ba-59e6d9a91ae9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:06.411723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:06.418170 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0ff409a0dd4e449e76998fb6591656', 'request_id': 'req-176bf5d4-3ded-4e61-a8f2-eff02883da58'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.425801 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0ff409a0dd4e449e76998fb6591656', 'request_id': 'req-176bf5d4-3ded-4e61-a8f2-eff02883da58'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.432135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0ff409a0dd4e449e76998fb6591656', 'request_id': 'req-176bf5d4-3ded-4e61-a8f2-eff02883da58'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.448107 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] The token cache is being invalidate because project 3b0ff409a0dd4e449e76998fb6591656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:06.448461 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.449987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.450936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1420/2841] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:06 2022] DELETE /identity/v3/projects/3b0ff409a0dd4e449e76998fb6591656 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:06.454936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:06.456880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.466406 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:06.467844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:06.468226 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:06.468628 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:06.470564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.537254 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bf5d4-3ded-4e61-a8f2-eff02883da58 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:06.542751 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:06.543501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:06.543847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:06.544190 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] PATH_INFO: `/v3/users/d31dcaa2f83e4118ab9bb989ab03e3c2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:06.546050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d31dcaa2f83e4118ab9bb989ab03e3c2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:06.548306 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:06.552578 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:06.555756 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:06.570147 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1422/2842] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:06.601260 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:06.602767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.620624 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd31dcaa2f83e4118ab9bb989ab03e3c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.622541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4d58-1b18-4bbb-8a1e-57e2f4324517 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:06.622681 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:06.629733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:06.631463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:06.632052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:06.632503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] PATH_INFO: `/v3/users/cb83494f073341019247c23545e801ff` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:06.634395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb83494f073341019247c23545e801ff)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:06.636967 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:06.650807 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd31dcaa2f83e4118ab9bb989ab03e3c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.659661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd31dcaa2f83e4118ab9bb989ab03e3c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.670075 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd31dcaa2f83e4118ab9bb989ab03e3c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.671191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0c288992-4c54-4220-8627-0d65dd1e361f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd31dcaa2f83e4118ab9bb989ab03e3c2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.679521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1421/2843] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:06 2022] DELETE /identity/v3/users/d31dcaa2f83e4118ab9bb989ab03e3c2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:06.729016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb83494f073341019247c23545e801ff'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.742929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb83494f073341019247c23545e801ff'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.778891 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb83494f073341019247c23545e801ff'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.780721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb83494f073341019247c23545e801ff'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.800951 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb83494f073341019247c23545e801ff'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:06.811228 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1423/2844] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:06 2022] DELETE /identity/v3/users/cb83494f073341019247c23545e801ff => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:06.816033 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:06.816348 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:06.816639 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:06.838955 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:06.840752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.922788 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1422/2845] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:06.934482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:06.937077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:06.995966 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:07.033894 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.035331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.037422 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147b19-0916-4026-8da7-310ad7f52a21 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.042779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.046630 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.047162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.047501 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.063392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde0a80-f0a7-46b5-bfa9-06e992d0abee None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.067039 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.068458 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.068768 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.068998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] PATH_INFO: `/v3/projects/d7c44120fdff48839380322afb5fbaa6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.069847 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.070182 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c44120fdff48839380322afb5fbaa6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.072067 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.072700 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.105021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1424/2846] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:10:07.147331 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c44120fdff48839380322afb5fbaa6', 'request_id': 'req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.155104 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c44120fdff48839380322afb5fbaa6', 'request_id': 'req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.161694 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c44120fdff48839380322afb5fbaa6', 'request_id': 'req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.181136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] The token cache is being invalidate because project d7c44120fdff48839380322afb5fbaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:07.181357 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.182093 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.185799 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1423/2847] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:07 2022] DELETE /identity/v3/projects/d7c44120fdff48839380322afb5fbaa6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:07.188723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.201611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bea33-9723-494a-a397-f6d12b3acea0 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.317988 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.318653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.318937 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.319197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] PATH_INFO: `/v3/users/f1ee1bbf4cbc47dfb458c808995ccb59` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.320941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ee1bbf4cbc47dfb458c808995ccb59)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.322989 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.399232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ee1bbf4cbc47dfb458c808995ccb59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.408655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ee1bbf4cbc47dfb458c808995ccb59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.428063 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ee1bbf4cbc47dfb458c808995ccb59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.429141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ee1bbf4cbc47dfb458c808995ccb59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.449071 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d66cddfe-727f-4df9-9a45-56e980613b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ee1bbf4cbc47dfb458c808995ccb59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.461371 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1425/2848] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:07 2022] DELETE /identity/v3/users/f1ee1bbf4cbc47dfb458c808995ccb59 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:07.476098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.477895 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.494880 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956ab7b-7b0e-4daa-85f6-4024e66ba22a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.557940 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:07.572806 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.573447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.573728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.574001 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] PATH_INFO: `/v3/projects/f3ecef1d1c9c40eaa48215e4000d6f5c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.575485 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3ecef1d1c9c40eaa48215e4000d6f5c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.578434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.629198 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3ecef1d1c9c40eaa48215e4000d6f5c', 'request_id': 'req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.635623 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3ecef1d1c9c40eaa48215e4000d6f5c', 'request_id': 'req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.644091 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3ecef1d1c9c40eaa48215e4000d6f5c', 'request_id': 'req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.661508 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] The token cache is being invalidate because project f3ecef1d1c9c40eaa48215e4000d6f5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:07.661936 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.662794 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9668dea8-b8e7-4eae-8c2d-4b3e96a67fbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.664116 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1424/2849] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:07 2022] DELETE /identity/v3/projects/f3ecef1d1c9c40eaa48215e4000d6f5c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:07.671227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.671518 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.671796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.691209 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:07.692659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.741969 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:07.756866 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1426/2850] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:07.764829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.764962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.765215 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.772069 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.773676 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.787723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699af2e9-6369-480c-bc86-992116cdb59b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.788444 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:07.789643 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.792912 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.793622 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.793975 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.794317 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] PATH_INFO: `/v3/projects/65b203e6701147139a1ad61e7a63000f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.796386 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b203e6701147139a1ad61e7a63000f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.798418 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.798991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1425/2851] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:07.803237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.813733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe729533-95a5-4cfa-8b74-5f9100dbc616 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.819013 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.819609 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.819885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.820152 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.820824 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:10:07.821126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.822710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.835391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1426/2852] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:10:07.839575 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.850595 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73258aba-0840-459e-b8a5-eb5f6bf2c019 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:07.851773 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b203e6701147139a1ad61e7a63000f', 'request_id': 'req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.858270 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b203e6701147139a1ad61e7a63000f', 'request_id': 'req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.865037 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b203e6701147139a1ad61e7a63000f', 'request_id': 'req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.878352 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] The token cache is being invalidate because project 65b203e6701147139a1ad61e7a63000f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:07.878639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.879280 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cd6f6fbd-1014-40db-8056-c5f5f0e6e3c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:07.880923 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1427/2853] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:07 2022] DELETE /identity/v3/projects/65b203e6701147139a1ad61e7a63000f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:07.885597 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8080c-0ff4-4906-b958-572c19f6a7bf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.885887 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8080c-0ff4-4906-b958-572c19f6a7bf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.886158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8080c-0ff4-4906-b958-572c19f6a7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.906538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-20b8080c-0ff4-4906-b958-572c19f6a7bf None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:07.907735 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-20b8080c-0ff4-4906-b958-572c19f6a7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.913753 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:07.914365 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.914648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.914922 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.915547 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:07.917618 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:07.949428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1427/2854] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:10:07 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:10:07.954467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:07.956078 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:07.967199 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1428/2855] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:07.974875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:07.975163 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:07.975440 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:07.994243 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:07.995430 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:08.001602 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:08.023448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ba261-7d54-4150-9736-bb30eb864dff admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.028187 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.028854 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.029207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.029484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.030115 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.032025 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.053957 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1428/2856] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:10:07 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:08.058002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.060002 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1429/2857] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:08.064513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.071741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eca596-06da-4402-8564-bd89a12ba114 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.073473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21ce97-a01a-4396-847d-75f4095e9955 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.076424 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.077029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.077311 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.077583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.078218 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.078661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.079350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.079701 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.079798 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.080103 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.081085 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:10:08.081463 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.083346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.090282 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1429/2858] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:10:08.094387 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.099264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1430/2859] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 13:10:08.103437 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.106687 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285fcb8-34a9-45ab-bb2a-9d35942cb241 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.111953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.112628 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.113020 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.113302 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.113837 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9a81-a4bb-4d88-9c8d-59aaa29f3334 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.113962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.115520 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.118392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.118981 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.119259 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.119531 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.120151 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.122058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.127558 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1430/2860] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:10:08.128107 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:08.131893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.141892 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf698117-267c-478b-b6af-05c0ab9630c0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.147972 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b872000-ca9c-4578-b03c-2da0e91ac5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.148654 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b872000-ca9c-4578-b03c-2da0e91ac5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.148939 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b872000-ca9c-4578-b03c-2da0e91ac5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.149214 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b872000-ca9c-4578-b03c-2da0e91ac5ba admin admin] PATH_INFO: `/v3/projects/8bec5f3334484051b6b6f4c0ee848e8f/users/e23717cb21e24f7cb23ec936b90ffd42/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.153781 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1431/2861] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:10:08 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:10:08.158566 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.164986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b872000-ca9c-4578-b03c-2da0e91ac5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bec5f3334484051b6b6f4c0ee848e8f, user_id=e23717cb21e24f7cb23ec936b90ffd42, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.167659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b872000-ca9c-4578-b03c-2da0e91ac5ba admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.180938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1431/2862] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:10:08 2022] PUT /identity/v3/projects/8bec5f3334484051b6b6f4c0ee848e8f/users/e23717cb21e24f7cb23ec936b90ffd42/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:08.192648 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960b17-b556-45f5-973b-d6484714680b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.193266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960b17-b556-45f5-973b-d6484714680b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.193266 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7960b17-b556-45f5-973b-d6484714680b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.221407 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a7960b17-b556-45f5-973b-d6484714680b None None] MFA Rules not processed for user `e23717cb21e24f7cb23ec936b90ffd42`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:08.222632 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a7960b17-b556-45f5-973b-d6484714680b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:08.224008 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c9f90-7d30-454b-a226-af10232c14bb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.278852 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.280027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.280333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.280621 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.281235 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.283141 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.289125 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1432/2863] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:08.301515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7960b17-b556-45f5-973b-d6484714680b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.303029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a7960b17-b556-45f5-973b-d6484714680b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:08.307936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1432/2864] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:10:08 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 13:10:08.308903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.318513 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc13ddb-c29e-43e4-aa42-8eba9e0ef1ef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.323202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.323793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.324074 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.324426 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.325032 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.326661 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.336527 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1433/2865] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:10:08.340584 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.351140 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648d3c9-c3f7-4268-b8e4-c7dd54bd0f6e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.355849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.356473 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.356878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.357289 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.358642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.360237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.365889 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7960b17-b556-45f5-973b-d6484714680b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.369851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6c330-a111-477d-b6f6-afbe43072e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.370583 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6c330-a111-477d-b6f6-afbe43072e43 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.370739 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1434/2866] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:10:08.371124 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6c330-a111-477d-b6f6-afbe43072e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.371529 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6c330-a111-477d-b6f6-afbe43072e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.375577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.381092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6c330-a111-477d-b6f6-afbe43072e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.382617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6c330-a111-477d-b6f6-afbe43072e43 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.384946 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c0b58-6cc8-4211-89c0-d5b1c012cf0a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.390467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.391105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.391385 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.391658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] PATH_INFO: `/v3/projects/9d04252e755e423eaf19e824f1c8c516/users/b4ce1e64fa924abeb7a1e5292a3ca96a/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.394150 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1433/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:10:08.407233 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d04252e755e423eaf19e824f1c8c516, user_id=b4ce1e64fa924abeb7a1e5292a3ca96a, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.409947 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.422956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1435/2868] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:10:08 2022] PUT /identity/v3/projects/9d04252e755e423eaf19e824f1c8c516/users/b4ce1e64fa924abeb7a1e5292a3ca96a/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:08.427893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16156e99-46f2-4033-9da3-47aea381cb6a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.428421 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16156e99-46f2-4033-9da3-47aea381cb6a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.428711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16156e99-46f2-4033-9da3-47aea381cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.449647 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-16156e99-46f2-4033-9da3-47aea381cb6a None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:08.450941 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-16156e99-46f2-4033-9da3-47aea381cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:08.465186 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:08.514570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1434/2869] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:08.525687 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:08.527522 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:08.570452 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:08.590464 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f13ccc-8011-499a-a151-4aed4cbbd258 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:08.595056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:08.595662 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.595939 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.596206 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:08.606999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:08.609552 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:08.619876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1436/2870] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:08.998764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f8f91-471e-4ac6-8915-c914c2443960 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:08.999057 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f8f91-471e-4ac6-8915-c914c2443960 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:08.999334 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f8f91-471e-4ac6-8915-c914c2443960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:09.018622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-366f8f91-471e-4ac6-8915-c914c2443960 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:09.019846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-366f8f91-471e-4ac6-8915-c914c2443960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:09.088528 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1435/2871] 10.222.0.57 () {58 vars in 967 bytes} [Thu Sep 15 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:09.103760 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:09.105205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:09.178058 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45cae5-b804-4cef-94af-93e3e929b4ac service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:09.183039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:09.183641 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:09.183917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:09.184187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:09.193799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:09.195369 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:09.203421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1437/2872] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:10:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:10:09.305926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e9efa-3610-42be-bd22-2dee96bde9d3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:09.305926 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e9efa-3610-42be-bd22-2dee96bde9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:09.306553 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e9efa-3610-42be-bd22-2dee96bde9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:09.346371 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-783e9efa-3610-42be-bd22-2dee96bde9d3 None None] MFA Rules not processed for user `b4ce1e64fa924abeb7a1e5292a3ca96a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:09.347775 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-783e9efa-3610-42be-bd22-2dee96bde9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:09.429351 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1436/2873] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:09.440534 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:09.443102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:09.520758 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3705b3-d3eb-41de-b765-8f14a81024b5 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:09.526193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:09.526878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:09.527150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:09.527484 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:09.548183 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:09.549747 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:09.559903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1438/2874] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:09.768515 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a45d-d0c0-414a-9d5c-ff02d3586726 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:09.768812 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a45d-d0c0-414a-9d5c-ff02d3586726 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:09.769089 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a45d-d0c0-414a-9d5c-ff02d3586726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:09.796998 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8b93a45d-d0c0-414a-9d5c-ff02d3586726 None None] MFA Rules not processed for user `e23717cb21e24f7cb23ec936b90ffd42`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:09.798430 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8b93a45d-d0c0-414a-9d5c-ff02d3586726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:09.859652 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:09.868352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1437/2875] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:10.338395 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:10.340052 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:10.410625 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7253a-6b93-4a9a-abea-2fbc057e0e0d service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:10.415612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:10.416152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:10.416529 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:10.416799 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:10.418475 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:10.429929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:10.431485 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:10.437349 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:10.444114 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1439/2876] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:10.981324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:10.981925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:10.981925 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:11.001636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] MFA Rules not processed for user `b4ce1e64fa924abeb7a1e5292a3ca96a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:11.002919 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:11.011035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1438/2877] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:11.234050 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:11.235564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:11.255390 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe8983-ee6d-465b-be3b-47b46b87298e service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:11.260166 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:11.260899 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:11.261242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:11.261594 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] PATH_INFO: `/v3/users/0fa77b2d0ace4ab3af23ec0a10e23ea1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:11.262779 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa77b2d0ace4ab3af23ec0a10e23ea1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:11.264903 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:11.282832 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:11.293531 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878dbb5-c8c1-4ac5-99e9-776bb24774fd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:11.299847 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:11.300702 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:11.301070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:11.301405 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:11.303491 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:11.313903 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:11.314498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa77b2d0ace4ab3af23ec0a10e23ea1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.315518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:11.323409 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa77b2d0ace4ab3af23ec0a10e23ea1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.324605 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1439/2878] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:10:11.335793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa77b2d0ace4ab3af23ec0a10e23ea1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.336563 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa77b2d0ace4ab3af23ec0a10e23ea1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.345372 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa77b2d0ace4ab3af23ec0a10e23ea1'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.352769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1440/2879] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:11 2022] DELETE /identity/v3/users/0fa77b2d0ace4ab3af23ec0a10e23ea1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:11.580526 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:11.612542 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167a874-a854-4d3d-b614-39e731d15d50 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:11.687566 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:11.688250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:11.688559 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:11.688829 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] PATH_INFO: `/v3/projects/9472f5867b964202af54362d390c7509` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:11.690029 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9472f5867b964202af54362d390c7509)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:11.692324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:11.745728 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9472f5867b964202af54362d390c7509', 'request_id': 'req-a3b523aa-92ec-46d9-903a-bc637026252a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.752846 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9472f5867b964202af54362d390c7509', 'request_id': 'req-a3b523aa-92ec-46d9-903a-bc637026252a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.759668 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9472f5867b964202af54362d390c7509', 'request_id': 'req-a3b523aa-92ec-46d9-903a-bc637026252a'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.776210 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] The token cache is being invalidate because project 9472f5867b964202af54362d390c7509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:11.776727 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.777433 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3b523aa-92ec-46d9-903a-bc637026252a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:11.779250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1440/2880] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:11 2022] DELETE /identity/v3/projects/9472f5867b964202af54362d390c7509 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:19.881467 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:19.883029 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:19.959976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b679b-5234-406d-8241-66e2e5bcabdb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:20.024436 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:20.025046 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:20.025324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:20.025702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:20.027454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:20.036878 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:20.039262 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:20.040958 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:20.053568 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1441/2881] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:21.772213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:27.380317 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466ba6b-c88e-4c0a-8196-d23a34cc45d4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:27.380917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466ba6b-c88e-4c0a-8196-d23a34cc45d4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:27.380917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466ba6b-c88e-4c0a-8196-d23a34cc45d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:27.401994 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8466ba6b-c88e-4c0a-8196-d23a34cc45d4 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:27.403175 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8466ba6b-c88e-4c0a-8196-d23a34cc45d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:27.462839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1441/2882] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:10:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:27.475649 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:27.477162 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:27.546198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15b714-213a-428d-8707-73a86755e47f service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:27.551373 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:27.551976 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:27.552257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:27.552564 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:27.560883 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:27.562580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:27.570769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1442/2883] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:10:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:10:28.074251 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f8ac2-4752-43ee-816f-fb1d9c740bd9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:28.074546 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f8ac2-4752-43ee-816f-fb1d9c740bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:28.074828 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f8ac2-4752-43ee-816f-fb1d9c740bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:28.094638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-393f8ac2-4752-43ee-816f-fb1d9c740bd9 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:28.095856 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-393f8ac2-4752-43ee-816f-fb1d9c740bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:28.103692 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1442/2884] 10.222.0.57 () {58 vars in 967 bytes} [Thu Sep 15 13:10:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:28.118658 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:28.130130 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad643aec-dcb6-4e76-995d-64d58dd29292 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:28.135158 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:28.135755 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:28.136035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:28.136338 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:28.145283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:28.146858 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:28.154970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1443/2885] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:10:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:10:28.396610 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b5372-82e4-4ada-b5de-1632d45962e8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:28.397359 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b5372-82e4-4ada-b5de-1632d45962e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:28.397463 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b5372-82e4-4ada-b5de-1632d45962e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:28.416432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-145b5372-82e4-4ada-b5de-1632d45962e8 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:28.417621 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-145b5372-82e4-4ada-b5de-1632d45962e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:28.424661 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1443/2886] 10.222.0.57 () {58 vars in 1009 bytes} [Thu Sep 15 13:10:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:28.436324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:28.447972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72089fd4-4f7e-426d-83bb-c9b6fa5f4eb0 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:28.456333 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:28.456955 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:28.457232 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:28.457503 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:28.466788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:28.468403 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:28.476355 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1444/2887] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:10:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:10:28.801890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:28.802174 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:28.802452 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:28.822590 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:28.823782 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:28.831146 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1444/2888] 10.222.0.57 () {58 vars in 967 bytes} [Thu Sep 15 13:10:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:28.843038 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:28.854798 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f19832-9ba4-474f-9f14-f8ab968411b7 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:28.860370 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558a4c9-03f4-498d-a2b5-3345e03f9846 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:28.860960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558a4c9-03f4-498d-a2b5-3345e03f9846 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:28.861237 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558a4c9-03f4-498d-a2b5-3345e03f9846 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:28.861506 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558a4c9-03f4-498d-a2b5-3345e03f9846 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:28.869368 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558a4c9-03f4-498d-a2b5-3345e03f9846 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:28.870929 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558a4c9-03f4-498d-a2b5-3345e03f9846 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:28.879884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1445/2889] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:10:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 13:10:31.017409 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.019068 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:31.091717 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b590d-1d0b-4b9f-97b2-99c430755ed4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:31.097555 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:31.098209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.098495 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.098796 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] PATH_INFO: `/v3/users/e23717cb21e24f7cb23ec936b90ffd42` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.099971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e23717cb21e24f7cb23ec936b90ffd42)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:31.101934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:31.150323 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e23717cb21e24f7cb23ec936b90ffd42'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.163373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e23717cb21e24f7cb23ec936b90ffd42'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.176505 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e23717cb21e24f7cb23ec936b90ffd42'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.184137 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e23717cb21e24f7cb23ec936b90ffd42'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.184780 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e23717cb21e24f7cb23ec936b90ffd42'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.191027 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1445/2890] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:10:31 2022] DELETE /identity/v3/users/e23717cb21e24f7cb23ec936b90ffd42 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:31.195455 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.195945 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.196053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.217245 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:31.218425 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:31.279124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1446/2891] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:31.288890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.290276 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:31.296924 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:31.351568 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc2137-da19-401d-96f3-e89e8e3d81cf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:31.355205 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ac82a-8236-4383-9fd1-aea46b1ed8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:31.355825 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ac82a-8236-4383-9fd1-aea46b1ed8df service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.356108 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ac82a-8236-4383-9fd1-aea46b1ed8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.356418 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ac82a-8236-4383-9fd1-aea46b1ed8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.367962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ac82a-8236-4383-9fd1-aea46b1ed8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:31.367962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ac82a-8236-4383-9fd1-aea46b1ed8df service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:31.377733 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1446/2892] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 13:10:31.554204 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.555720 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:31.581872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fc29e-7dbb-4713-aaed-75bdd0903221 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:31.589970 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:31.590646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.590927 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.591193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] PATH_INFO: `/v3/projects/8bec5f3334484051b6b6f4c0ee848e8f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.592357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bec5f3334484051b6b6f4c0ee848e8f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:31.594263 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:31.635917 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:31.642635 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bec5f3334484051b6b6f4c0ee848e8f', 'request_id': 'req-af96e56d-a8ca-47d0-9b44-211360e6f326'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.648500 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bec5f3334484051b6b6f4c0ee848e8f', 'request_id': 'req-af96e56d-a8ca-47d0-9b44-211360e6f326'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.654744 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bec5f3334484051b6b6f4c0ee848e8f', 'request_id': 'req-af96e56d-a8ca-47d0-9b44-211360e6f326'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.667573 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] The token cache is being invalidate because project 8bec5f3334484051b6b6f4c0ee848e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:10:31.667867 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.668519 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-af96e56d-a8ca-47d0-9b44-211360e6f326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:10:31.670011 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1447/2893] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:10:31 2022] DELETE /identity/v3/projects/8bec5f3334484051b6b6f4c0ee848e8f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:31.675589 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.675883 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.676159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.697257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:31.698539 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:31.757836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1447/2894] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:31.765730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.766017 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.766294 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.787434 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:31.788681 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:31.796551 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1448/2895] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:31.800896 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.810667 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34d147-090f-474a-b10a-92e70c4cf4f4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:31.815521 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:31.816126 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.816453 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.816730 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.817380 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 13:10:31.817686 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:31.819269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:31.831207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1448/2896] 10.222.0.57 () {58 vars in 1153 bytes} [Thu Sep 15 13:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 13:10:31.835361 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.844449 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7e29-f981-49be-b97c-c797939dadee None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:31.851040 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:31.851860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.852257 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.852685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.853761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:31.856092 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:31.883842 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1449/2897] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:10:31 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:31.887990 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.899171 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b33ae7-5b51-4972-a263-d670963025b9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:31.959138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:31.959752 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:31.960035 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:31.960338 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:31.960971 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:31.962875 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:31.986898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1449/2898] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:10:31 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:31.988049 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:31.998247 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622515e5-9f6e-427b-afc1-c8f876f06c35 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:32.003060 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:32.003653 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:32.003930 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:32.004202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:32.004870 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:32.006454 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:32.017516 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1450/2899] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:10:32.022294 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:32.033127 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c2507-4956-4639-94df-295d86c19b10 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:32.037818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050ae89-9dfa-4956-9522-d867ba3d3a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:32.038414 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050ae89-9dfa-4956-9522-d867ba3d3a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:32.038697 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050ae89-9dfa-4956-9522-d867ba3d3a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:32.038969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050ae89-9dfa-4956-9522-d867ba3d3a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:32.039633 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050ae89-9dfa-4956-9522-d867ba3d3a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:32.041265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050ae89-9dfa-4956-9522-d867ba3d3a05 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:32.051836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1450/2900] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:10:32.056003 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:32.066580 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc910-8d92-4d67-a401-100d36f0d218 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:32.071096 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:32.071733 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:32.072015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:32.072313 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] PATH_INFO: `/v3/projects/1344373f29a44b7c8db2f9bab5f38467/users/3c50430b23404745ab4c1099bf12d4f5/roles/3901992ddd9a4c30b6faf511834b6e21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:32.089722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1344373f29a44b7c8db2f9bab5f38467, user_id=3c50430b23404745ab4c1099bf12d4f5, role_id=3901992ddd9a4c30b6faf511834b6e21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:32.093595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:32.108444 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1451/2901] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:10:32 2022] PUT /identity/v3/projects/1344373f29a44b7c8db2f9bab5f38467/users/3c50430b23404745ab4c1099bf12d4f5/roles/3901992ddd9a4c30b6faf511834b6e21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:10:32.113572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32e051-3279-4345-9635-9c735e7ce283 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:32.113872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32e051-3279-4345-9635-9c735e7ce283 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:32.114154 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32e051-3279-4345-9635-9c735e7ce283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:32.134591 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0f32e051-3279-4345-9635-9c735e7ce283 None None] MFA Rules not processed for user `07cb4dfbc4ad477d912360d9e55fab21`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:32.135803 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0f32e051-3279-4345-9635-9c735e7ce283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:32.145002 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:32.209259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1451/2902] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:32.218914 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:32.221603 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:32.280646 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5030a-6c76-4070-a7ae-dbbd30f1ebf6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:32.284392 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:32.285004 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:32.285285 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:32.285556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:32.294803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:32.296242 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:32.305788 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1452/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:36.472233 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.472757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.472857 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.510938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] MFA Rules not processed for user `3c50430b23404745ab4c1099bf12d4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:36.512176 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:36.604144 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1452/2904] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:10:36.605707 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:36.616793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecc666-318b-4a70-a457-ec6120fcb5b0 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:36.621743 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:36.622438 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.622787 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.623128 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.623918 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:36.626556 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:36.668998 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1453/2905] 10.222.0.57 () {60 vars in 1153 bytes} [Thu Sep 15 13:10:36 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:36.672898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:36.684622 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a0961-5438-493f-802f-08a0c1729896 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:36.744511 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:36.745239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.745587 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.745930 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.746635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:36.748684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:36.775392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1453/2906] 10.222.0.57 () {60 vars in 1144 bytes} [Thu Sep 15 13:10:36 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 13:10:36.779509 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:36.793723 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f384-7b73-42bb-87c2-8b8c4d568b3a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:36.799423 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:36.800187 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.800557 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.800950 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.801663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:36.803468 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:36.818053 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1454/2907] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 13:10:36.822270 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:36.832991 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b89ed-0a2d-46a0-82b8-978a87e1b511 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:36.837767 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:36.838432 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.838714 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.838987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] PATH_INFO: `/v3/projects/4c6e6a2258c245e99e594f9da66f1610/users/aa615468d95d48ff88a8d6ebb95a0d61/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.856861 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6e6a2258c245e99e594f9da66f1610, user_id=aa615468d95d48ff88a8d6ebb95a0d61, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:36.859501 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:36.873890 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1454/2908] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:10:36 2022] PUT /identity/v3/projects/4c6e6a2258c245e99e594f9da66f1610/users/aa615468d95d48ff88a8d6ebb95a0d61/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:36.878100 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:36.888420 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd28f9-48c3-4012-afd8-90437767184c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:36.922541 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:36.948598 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d111b-7c42-459b-83c3-94369350d6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:36.949208 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d111b-7c42-459b-83c3-94369350d6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.949489 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d111b-7c42-459b-83c3-94369350d6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.949761 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d111b-7c42-459b-83c3-94369350d6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.950398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d111b-7c42-459b-83c3-94369350d6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:36.951986 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d111b-7c42-459b-83c3-94369350d6b1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:36.962685 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1455/2909] 10.222.0.57 () {58 vars in 1122 bytes} [Thu Sep 15 13:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 13:10:36.966967 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:36.978319 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc235bd-ed2a-4a88-b4c1-4669e07ca165 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:36.983200 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:36.983872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:36.984159 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:36.984464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] PATH_INFO: `/v3/projects/4c6e6a2258c245e99e594f9da66f1610/users/aa615468d95d48ff88a8d6ebb95a0d61/roles/13d4313b2fe74b3688999f3077afce5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:36.986496 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6e6a2258c245e99e594f9da66f1610, user_id=aa615468d95d48ff88a8d6ebb95a0d61, role_id=13d4313b2fe74b3688999f3077afce5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:36.989250 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:36.992718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:37.002046 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1455/2910] 10.222.0.57 () {60 vars in 1483 bytes} [Thu Sep 15 13:10:36 2022] PUT /identity/v3/projects/4c6e6a2258c245e99e594f9da66f1610/users/aa615468d95d48ff88a8d6ebb95a0d61/roles/13d4313b2fe74b3688999f3077afce5d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:10:41.991337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566f8a-b9ac-4469-bf17-6da56d672c51 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:41.991916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566f8a-b9ac-4469-bf17-6da56d672c51 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:41.991916 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566f8a-b9ac-4469-bf17-6da56d672c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:42.027035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-77566f8a-b9ac-4469-bf17-6da56d672c51 None None] MFA Rules not processed for user `aa615468d95d48ff88a8d6ebb95a0d61`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:10:42.028168 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-77566f8a-b9ac-4469-bf17-6da56d672c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:42.123424 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1456/2911] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:10:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:10:42.131691 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:10:42.135112 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:10:42.217342 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa37e8-9d62-4005-be9e-aaae11dedefb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:10:42.224866 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b49a-6c17-4900-a44f-d700e8216309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:10:42.225611 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b49a-6c17-4900-a44f-d700e8216309 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:10:42.226333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b49a-6c17-4900-a44f-d700e8216309 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:10:42.226333 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b49a-6c17-4900-a44f-d700e8216309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:10:42.292369 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516b49a-6c17-4900-a44f-d700e8216309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:10:42.293962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516b49a-6c17-4900-a44f-d700e8216309 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:10:42.304964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1456/2912] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:10:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:10:52.112435 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=47 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:10:52.300136 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=58 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:11:18.673631 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:11:18.674892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:11:18.674892 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:11:18.693188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] MFA Rules not processed for user `3c50430b23404745ab4c1099bf12d4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:11:18.694376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:11:18.701156 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1457/2913] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:11:28.687741 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:11:37.799301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:11:37.800503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:11:37.800503 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:11:37.818976 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] MFA Rules not processed for user `b4ce1e64fa924abeb7a1e5292a3ca96a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:11:37.820193 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:11:37.877634 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1457/2914] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:11:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:11:38.122525 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:11:38.185504 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c277a1b-332a-44e8-a03e-8b86f70896f6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:11:38.189150 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:11:38.190339 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:11:38.191421 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:11:38.191791 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:11:38.193611 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:11:38.203521 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:11:38.205001 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:11:38.213775 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1458/2915] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:11:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:11:40.301467 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:11:40.303032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:11:40.316237 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649dec0-b953-49ba-818f-2454aaa6f38c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:11:40.319760 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654f90b-16ed-4034-b8ac-f33b088369be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:11:40.320455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654f90b-16ed-4034-b8ac-f33b088369be service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:11:40.320735 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654f90b-16ed-4034-b8ac-f33b088369be service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:11:40.321005 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654f90b-16ed-4034-b8ac-f33b088369be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:11:40.330164 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654f90b-16ed-4034-b8ac-f33b088369be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:11:40.331969 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654f90b-16ed-4034-b8ac-f33b088369be service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:11:40.341474 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1458/2916] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:11:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:11:48.210115 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=26 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:11:50.290984 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:11:50.292497 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:11:50.337512 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=31 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:11:50.371401 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41443fb5-fd30-4d2f-b380-15d3b0762833 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:11:50.376595 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:11:50.377202 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:11:50.377480 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:11:50.377753 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:11:50.379672 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:11:50.391771 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:11:50.393398 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:11:50.405399 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1459/2917] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:11:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:12:00.400742 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=32 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:12:20.696318 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:20.696860 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:20.697110 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:20.717644 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] MFA Rules not processed for user `61362637fe434cbeba9d2d9efe37605a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:12:20.718844 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:20.778204 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1459/2918] 10.222.0.57 () {58 vars in 967 bytes} [Thu Sep 15 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:12:20.827614 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:20.829056 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:20.908456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac6a0e-3276-4295-b785-5049c46a9ab7 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:20.914642 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ead9001bd3241399dac6579a9588975', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:20.915565 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:20.915849 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:20.916125 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:20.925082 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:20.926659 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:20.936493 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1460/2919] 10.222.0.57 () {60 vars in 1384 bytes} [Thu Sep 15 13:12:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 13:12:24.432964 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:24.445764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b46d64-738f-4084-a798-04b2320dba87 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:24.451257 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:24.451962 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:24.452361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:24.452707 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:24.454761 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:24.468572 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:24.470185 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:24.489787 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1460/2920] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:12:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:12:27.631135 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:27.632762 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:27.714884 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cfa40-8e65-40fe-a5d7-9cc30e3b11ed service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:27.719538 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:27.720176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:27.720482 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:27.720933 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] PATH_INFO: `/v3/users/fd09d576051f4b98a8a9b3ec0ca18bc8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:27.722227 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd09d576051f4b98a8a9b3ec0ca18bc8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:27.725337 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:27.801928 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd09d576051f4b98a8a9b3ec0ca18bc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:27.809948 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd09d576051f4b98a8a9b3ec0ca18bc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:27.822840 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd09d576051f4b98a8a9b3ec0ca18bc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:27.823555 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd09d576051f4b98a8a9b3ec0ca18bc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:27.831027 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd09d576051f4b98a8a9b3ec0ca18bc8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:27.838477 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1461/2921] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:12:27 2022] DELETE /identity/v3/users/fd09d576051f4b98a8a9b3ec0ca18bc8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:12:28.012831 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:28.026468 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5995e2e-7875-462b-a432-96bb9c911b5b service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:28.088034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:28.088708 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:28.088988 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:28.089260 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] PATH_INFO: `/v3/projects/8b815e753bbe4a54942550a564653fd8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:28.090373 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b815e753bbe4a54942550a564653fd8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:28.092324 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:28.138439 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b815e753bbe4a54942550a564653fd8', 'request_id': 'req-4406609a-df94-42b6-9e1f-131efb9e84d7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.143743 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b815e753bbe4a54942550a564653fd8', 'request_id': 'req-4406609a-df94-42b6-9e1f-131efb9e84d7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.151052 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b815e753bbe4a54942550a564653fd8', 'request_id': 'req-4406609a-df94-42b6-9e1f-131efb9e84d7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.163434 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] The token cache is being invalidate because project 8b815e753bbe4a54942550a564653fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:12:28.163718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.164364 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.165836 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1461/2922] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:12:28 2022] DELETE /identity/v3/projects/8b815e753bbe4a54942550a564653fd8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:12:28.169819 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:28.171193 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:28.185055 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b855b-5904-4ebc-9d5f-e41aaa784cb9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:28.189510 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:12:28.243035 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:28.243663 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:28.243941 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:28.244214 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] PATH_INFO: `/v3/users/c5502530f4dc4fd8972321895a30f5f2` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:28.245366 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5502530f4dc4fd8972321895a30f5f2)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:28.247282 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:28.296496 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5502530f4dc4fd8972321895a30f5f2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.305068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5502530f4dc4fd8972321895a30f5f2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.316789 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5502530f4dc4fd8972321895a30f5f2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.317472 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5502530f4dc4fd8972321895a30f5f2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.325532 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5502530f4dc4fd8972321895a30f5f2'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.331466 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1462/2923] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:12:28 2022] DELETE /identity/v3/users/c5502530f4dc4fd8972321895a30f5f2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:12:28.503092 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:28.504617 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:28.520295 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406609a-df94-42b6-9e1f-131efb9e84d7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:28.539997 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:12:28.579326 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:28.579958 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:28.580243 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:28.580534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] PATH_INFO: `/v3/projects/de7356a41c984673bc0591b18f223208` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:28.581635 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7356a41c984673bc0591b18f223208)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:28.584151 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:28.629050 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7356a41c984673bc0591b18f223208', 'request_id': 'req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.634771 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7356a41c984673bc0591b18f223208', 'request_id': 'req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.640938 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7356a41c984673bc0591b18f223208', 'request_id': 'req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.653818 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] The token cache is being invalidate because project de7356a41c984673bc0591b18f223208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:12:28.654109 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.654744 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:28.656225 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1462/2924] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:12:28 2022] DELETE /identity/v3/projects/de7356a41c984673bc0591b18f223208 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 13:12:38.328124 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:12:38.650804 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:12:46.615879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:46.617375 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:46.630456 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f9737-a598-4826-b904-efa2c8fe84e1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:46.685721 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:46.686346 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:46.686633 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:46.686908 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] PATH_INFO: `/v3/users/b4ce1e64fa924abeb7a1e5292a3ca96a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:46.688021 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ce1e64fa924abeb7a1e5292a3ca96a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:46.689960 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:46.737367 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ce1e64fa924abeb7a1e5292a3ca96a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:46.745632 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ce1e64fa924abeb7a1e5292a3ca96a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:46.759059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ce1e64fa924abeb7a1e5292a3ca96a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:46.759872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ce1e64fa924abeb7a1e5292a3ca96a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:46.767882 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ce1e64fa924abeb7a1e5292a3ca96a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:46.776309 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1463/2925] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:12:46 2022] DELETE /identity/v3/users/b4ce1e64fa924abeb7a1e5292a3ca96a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:12:46.960249 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:12:46.961898 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:12:46.981465 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc9cd0-5fd7-4fa1-a47b-3813983a18a4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:12:47.039239 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:12:47.039872 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:12:47.040148 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:12:47.040450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] PATH_INFO: `/v3/projects/9d04252e755e423eaf19e824f1c8c516` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:12:47.041557 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d04252e755e423eaf19e824f1c8c516)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:12:47.043451 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:12:47.091627 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d04252e755e423eaf19e824f1c8c516', 'request_id': 'req-3d207dd7-0869-40e9-a3a6-daba609565b3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:47.101755 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d04252e755e423eaf19e824f1c8c516', 'request_id': 'req-3d207dd7-0869-40e9-a3a6-daba609565b3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:47.108034 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d04252e755e423eaf19e824f1c8c516', 'request_id': 'req-3d207dd7-0869-40e9-a3a6-daba609565b3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:47.121095 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] The token cache is being invalidate because project 9d04252e755e423eaf19e824f1c8c516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:12:47.121385 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:47.122010 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:12:47.123711 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1463/2926] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:12:46 2022] DELETE /identity/v3/projects/9d04252e755e423eaf19e824f1c8c516 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:12:56.770323 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:12:57.117885 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:14:21.254283 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:14:21.255879 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:21.312357 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd24b-375a-4994-8323-0dd8e4813d5f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:14:21.359044 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:14:21.359655 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:14:21.359936 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:14:21.360211 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:14:21.361953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:21.432863 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:14:21.434324 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:14:21.445196 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1464/2927] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:14:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:14:22.635391 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:14:22.636909 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:22.649978 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d207dd7-0869-40e9-a3a6-daba609565b3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:14:22.653534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:14:22.654135 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:14:22.654413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:14:22.654684 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:14:22.656392 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:22.669986 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:14:22.671399 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab19bb2-2390-4e23-ad67-dd6e53d4086e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:14:22.683259 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1464/2928] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:14:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:14:24.175272 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:14:24.177116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:24.195767 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a429d-2bb1-4051-91fb-89cf7d83f197 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:14:24.204039 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:14:24.204796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:14:24.205197 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:14:24.205524 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:14:24.207413 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:24.221198 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:14:24.222856 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:14:24.235116 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1465/2929] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:14:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:14:26.105781 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:14:26.106229 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:14:26.106459 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:14:26.126209 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] MFA Rules not processed for user `3c50430b23404745ab4c1099bf12d4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:14:26.128636 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:26.186441 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1465/2930] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:14:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 13:14:28.689171 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:14:28.700264 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0f132-3f6e-4286-8e6a-cb75f0a87e58 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:14:28.705790 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:14:28.706796 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:14:28.707155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:14:28.707577 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:14:28.709508 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:14:28.721638 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:14:28.723188 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1b954-dd79-4c1f-9226-cbfa6d1f57d1 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:14:28.732639 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1466/2931] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:14:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:14:36.179719 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=31 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:14:38.728812 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:16:31.815678 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:16:31.817142 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:31.874070 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dcc8a-8f5e-4234-818c-7aa85d93c927 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:16:31.877851 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d2458e0ed4b4fceb437ac5c7b43d598', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:16:31.878464 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:31.878748 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:31.879024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:31.880777 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:31.892419 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:16:31.893839 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:16:31.903638 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1466/2932] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:16:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:16:41.896712 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:41.897730 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:41.897844 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:41.899604 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=26 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:16:41.917220 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] MFA Rules not processed for user `3c50430b23404745ab4c1099bf12d4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 13:16:41.918414 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:41.925176 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1467/2933] 10.222.0.57 () {58 vars in 958 bytes} [Thu Sep 15 13:16:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 13:16:42.261448 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:16:42.262837 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:42.274616 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b6a3c-654f-4078-b80c-36860d329b4d service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:16:42.278121 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:16:42.278712 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:42.278987 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:42.279253 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:42.280893 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:42.290552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:16:42.291943 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:16:42.299810 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1467/2934] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:16:42.744105 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:16:42.745559 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:42.817822 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe251c5-1911-45b0-b09a-ae1456eeed1d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:16:42.822713 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:16:42.823321 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:42.823600 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:42.823873 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:42.825654 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:42.837574 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:16:42.839152 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:16:42.848860 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1468/2935] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:16:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:16:52.296168 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:16:52.844616 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=35 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:16:56.205778 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:16:56.207199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:56.293659 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6907cb2-58e3-4c7d-b64f-25bbbd9b79cc service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:16:56.298757 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:16:56.299361 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:56.299642 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:56.299914 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:56.301655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:56.313742 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:16:56.315248 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:16:56.325953 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1468/2936] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:16:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:16:56.952077 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:16:57.011248 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a178ce-fcb0-47bb-a1ef-3c3cd458f4c5 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:16:57.017460 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:16:57.018068 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:57.018972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:57.018972 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:57.020612 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:57.033793 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:16:57.035376 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:16:57.046015 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1469/2937] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:16:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:16:59.529213 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:16:59.548517 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ee38-76b8-4bde-8c21-bf0b5d766426 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:16:59.552098 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:16:59.552737 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:16:59.553024 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:16:59.553303 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:16:59.555570 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:16:59.618049 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:16:59.619413 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:16:59.628944 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1469/2938] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:16:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:17:01.397705 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:17:01.409143 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bdb09-9c1a-4f01-854d-8351e80af55c service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:17:01.415104 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03f7ea8f3fb64e3d96fbe277e07b5435', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:17:01.415702 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:17:01.415978 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:17:01.416246 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:17:01.418016 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:17:01.431072 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:17:01.432666 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:17:01.443459 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1470/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:17:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:17:09.624699 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=71 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:17:11.438872 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=26 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:19:03.515959 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:19:03.525518 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039f09-db77-48f9-a3ce-d351d00ab9a3 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:19:03.529191 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '53860f84e5cc4c84b15a8340f10c26d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:19:03.529929 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:19:03.530212 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:19:03.530484 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:19:03.532263 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:19:03.617138 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:19:03.618551 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:19:03.636935 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1470/2940] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:19:06.433670 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:19:06.446344 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c38b2-caeb-40bb-8043-37f5dfc077c0 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:19:06.451722 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:19:06.452447 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:19:06.452831 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:19:06.453102 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:19:06.455330 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:19:06.473931 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:19:06.477814 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:19:06.498851 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1471/2941] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:19:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:19:09.577718 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:19:09.579184 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:19:09.591455 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a740a1-83dc-4a87-8976-fa15197ce8ec service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:19:09.595966 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:19:09.596655 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:19:09.596934 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:19:09.597207 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] PATH_INFO: `/v3/users/3c50430b23404745ab4c1099bf12d4f5` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:19:09.598315 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c50430b23404745ab4c1099bf12d4f5)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:19:09.600255 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:19:09.647830 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c50430b23404745ab4c1099bf12d4f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.659710 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c50430b23404745ab4c1099bf12d4f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.666764 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c50430b23404745ab4c1099bf12d4f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.673917 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c50430b23404745ab4c1099bf12d4f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.674534 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c50430b23404745ab4c1099bf12d4f5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.681269 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1471/2942] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:19:09 2022] DELETE /identity/v3/users/3c50430b23404745ab4c1099bf12d4f5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:19:09.855769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:19:09.871302 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c4b6f-d3d8-4d90-891d-6bfce64e5b93 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:19:09.930956 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:19:09.931589 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:19:09.931875 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:19:09.932155 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] PATH_INFO: `/v3/projects/1344373f29a44b7c8db2f9bab5f38467` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:19:09.933305 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1344373f29a44b7c8db2f9bab5f38467)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:19:09.935210 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:19:09.983581 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1344373f29a44b7c8db2f9bab5f38467', 'request_id': 'req-4a663daa-0702-4977-b37c-a0f2f09e8cc9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.989254 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1344373f29a44b7c8db2f9bab5f38467', 'request_id': 'req-4a663daa-0702-4977-b37c-a0f2f09e8cc9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:09.996205 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1344373f29a44b7c8db2f9bab5f38467', 'request_id': 'req-4a663daa-0702-4977-b37c-a0f2f09e8cc9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.009059 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] The token cache is being invalidate because project 1344373f29a44b7c8db2f9bab5f38467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:19:10.009353 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.009997 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.011498 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1472/2943] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:19:09 2022] DELETE /identity/v3/projects/1344373f29a44b7c8db2f9bab5f38467 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:19:10.015631 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:19:10.017797 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:19:10.032032 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fdb3e-b7cb-4df1-918b-5e06648fdbac admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:19:10.067265 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:19:10.096447 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:19:10.097102 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:19:10.097384 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:19:10.097656 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] PATH_INFO: `/v3/users/aa615468d95d48ff88a8d6ebb95a0d61` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:19:10.098811 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa615468d95d48ff88a8d6ebb95a0d61)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:19:10.101301 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:19:10.149352 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa615468d95d48ff88a8d6ebb95a0d61'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.161199 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa615468d95d48ff88a8d6ebb95a0d61'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.167920 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa615468d95d48ff88a8d6ebb95a0d61'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.175044 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa615468d95d48ff88a8d6ebb95a0d61'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.175658 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa615468d95d48ff88a8d6ebb95a0d61'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.182220 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1472/2944] 10.222.0.57 () {58 vars in 1224 bytes} [Thu Sep 15 13:19:10 2022] DELETE /identity/v3/users/aa615468d95d48ff88a8d6ebb95a0d61 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:19:10.353439 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:19:10.354876 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:19:10.367769 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a663daa-0702-4977-b37c-a0f2f09e8cc9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:19:10.424490 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07cb4dfbc4ad477d912360d9e55fab21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53ac1f185c9948448574c644790cea74', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:19:10.425122 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:19:10.425432 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:19:10.425727 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] PATH_INFO: `/v3/projects/4c6e6a2258c245e99e594f9da66f1610` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:19:10.426838 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6e6a2258c245e99e594f9da66f1610)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:19:10.428999 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:19:10.450219 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:19:10.476797 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6e6a2258c245e99e594f9da66f1610', 'request_id': 'req-6c65c726-f694-4df2-b22d-2c5a573aaed9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.482065 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6e6a2258c245e99e594f9da66f1610', 'request_id': 'req-6c65c726-f694-4df2-b22d-2c5a573aaed9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.487935 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6e6a2258c245e99e594f9da66f1610', 'request_id': 'req-6c65c726-f694-4df2-b22d-2c5a573aaed9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.499953 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] The token cache is being invalidate because project 4c6e6a2258c245e99e594f9da66f1610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 13:19:10.500241 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.500901 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c65c726-f694-4df2-b22d-2c5a573aaed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 13:19:10.502350 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1473/2945] 10.222.0.57 () {58 vars in 1233 bytes} [Thu Sep 15 13:19:10 2022] DELETE /identity/v3/projects/4c6e6a2258c245e99e594f9da66f1610 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 13:19:20.178498 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:19:20.496803 dv-cinder-836083-13-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 13:21:59.192620 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 13:21:59.194277 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:21:59.263552 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cb8f6-afaf-41c0-9ebd-ed19779dfa38 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 13:21:59.322565 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25816463963b44a8865c0d3693cfef88', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '99e6a332ee23460796823d362d81e7bd', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 13:21:59.323172 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 13:21:59.323450 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 13:21:59.323716 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 13:21:59.325428 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 13:21:59.336731 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 13:21:59.338288 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ed88d-15a7-49f9-95b8-2f58882b25b8 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 13:21:59.348042 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1473/2946] 10.222.0.57 () {60 vars in 1365 bytes} [Thu Sep 15 13:21:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 13:22:09.343741 dv-cinder-836083-13-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=58 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}